-- Logs begin at Thu 2022-09-08 12:46:52 UTC, end at Thu 2022-09-08 14:46:25 UTC. -- Sep 08 13:31:59.490861 dv-cinder-856460-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 08 13:31:59.500083 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 13:31:59 2022] *** Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: nodename: dv-cinder-856460-1-iscsi Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: machine: x86_64 Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: clock source: unix Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: pcre jit disabled Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: detected number of CPU cores: 4 Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: current working directory: / Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: detected binary path: /usr/bin/uwsgi-core Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: your processes number limit is 47680 Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: your memory page size is 4096 bytes Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: detected max file descriptor number: 2048 Sep 08 13:31:59.502995 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: lock engine: pthread robust mutexes Sep 08 13:31:59.503781 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: thunder lock: enabled Sep 08 13:31:59.503781 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 08 13:31:59.503781 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 13:31:59.542823 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: Python main interpreter initialized at 0x561bd7ad69c0 Sep 08 13:31:59.542823 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: python threads support enabled Sep 08 13:31:59.542823 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: your server socket listen backlog is limited to 100 connections Sep 08 13:31:59.542823 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: your mercy for graceful operations on workers is 90 seconds Sep 08 13:31:59.543155 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 13:31:59.543232 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: *** Operational MODE: preforking *** Sep 08 13:31:59.543504 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: *** uWSGI is running in multiple interpreter mode *** Sep 08 13:31:59.543504 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: spawned uWSGI master process (pid: 88244) Sep 08 13:31:59.543705 dv-cinder-856460-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 08 13:31:59.546273 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: spawned uWSGI worker 1 (pid: 88255, cores: 1) Sep 08 13:31:59.546273 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: spawned uWSGI worker 2 (pid: 88256, cores: 1) Sep 08 13:31:59.546273 dv-cinder-856460-1-iscsi devstack@keystone.service[88244]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 13:32:03.931601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:32:03.931601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: from cryptography.utils import int_from_bytes Sep 08 13:32:03.932372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:32:03.932372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: from cryptography.utils import int_from_bytes Sep 08 13:32:03.933202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:32:03.933202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: from cryptography.utils import int_from_bytes Sep 08 13:32:03.933709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:32:03.933709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: from cryptography.utils import int_from_bytes Sep 08 13:32:03.983804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:32:03.984019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:32:03.984171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] command line args: [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:32:03.984441 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:32:03.984616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ================================================================================ {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:32:03.984872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] admin_token = **** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.985114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] config_dir = [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.985301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.985481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] config_source = [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.985660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] debug = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.985861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.986260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.986441 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.986691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.986942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.987171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] list_limit = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.987348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] log_config_append = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.987525 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.987701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] log_dir = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.987875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] log_file = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.988109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.988367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.988597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.988846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.989022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.989324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.989582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.989828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.990063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.990320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.990555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.990787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.991031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.991277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.991552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.991996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:32:03.992077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.992192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:32:03.992273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] publish_errors = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.992382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] command line args: [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:32:03.992460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.992655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:32:03.992741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.992833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.992908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ================================================================================ {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:32:03.993080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.993158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] admin_token = **** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.993316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.993394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] config_dir = [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.993821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.993923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] standard_threads = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] config_source = [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] debug = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] use_journal = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] use_json = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] use_stderr = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.994988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.995065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] use_syslog = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.995136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.995215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.995290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] list_limit = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.995368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.995439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] log_config_append = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.995540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.995622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.995719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] log_dir = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.995797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.995900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] log_file = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.996024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.996136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.996272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.996392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.996641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.996789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.996892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.997071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.997234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.997319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.997478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.997560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.997721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.997803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.997967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.998061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:03.998238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:03.998320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.005580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.005744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.005744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.005947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.005947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.006096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.006200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.006275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.006344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.006426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.006499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.006499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.006637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.006708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.006855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] publish_errors = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.006929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.006999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.007068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.007144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.007225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.007302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.007374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.007443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.007443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.007566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.007646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] standard_threads = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.007723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.007798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.007874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.007945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.008022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.008093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.008167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.008246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] use_journal = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.008324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.008442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] use_json = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.008528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.008604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] use_stderr = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.008604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] use_syslog = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.008729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.008798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:32:04.008867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.008943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.009993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.010974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.011982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012105 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012774 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.012963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.013903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.014880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.015115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.015239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.016086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.016268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.016339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.016571 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.016647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.016814 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.017031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.017098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.017287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.017419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.017530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.017727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.017805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.017999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.018066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.018255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.018379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.018514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.018754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.018821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.018994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.019119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.019229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.019376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.019599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.019723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.019831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.019935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.020977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.021935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.022043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.022210 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.022323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.022434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.022548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.022654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.023201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.023358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.023482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.023595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.023745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.023869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.023950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] auth.external = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.024978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.025136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.025283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.025388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.025466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] auth.password = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.025655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.025751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] auth.token = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.025891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.025967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.026101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.026245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.026366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.026462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.026595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.026662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.026839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.026914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.027073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.027160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.027303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.027388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.027534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.027611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] credential.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.027773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.027850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.028011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.028124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.028290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.028366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.028539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.028657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.028783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.028878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.029018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.029108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.029260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.029342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.029512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.029613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.029775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.029900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.030017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] auth.external = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.030096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.030331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.030426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.030578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.030657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.030840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.030918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.031081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.031168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] auth.password = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.031327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] auth.token = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.031405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.031559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] federation.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.031678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.031795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.031884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.032038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.032146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.032282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.032384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.032513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.032684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.032764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.032940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] credential.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.033984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.034190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.034275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.034400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.034479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.034646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.039981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.040922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] federation.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.041962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.042927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.043001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.043001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.043119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.043459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.043754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.043841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.043915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.043987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044344 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.044968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.045901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046639 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.046986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.047946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.048945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.049962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.050073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.050173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.050329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.050410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.050576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.050657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.050812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.050891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.051048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.051126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.051281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.051390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.051515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.051629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.051758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.051864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.052982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.053160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.053284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.053393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.053525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.053627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.053774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.053854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.054040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.054120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.054300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.054407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.054552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.054632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.054789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] resource.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.054868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.055986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.056068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.056217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.056375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.056454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.056527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.056742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.056829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.056956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] role.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] role.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.057990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.058168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.058253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.058414 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.058512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.058654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.058736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.058895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.058976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.059139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.059228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.059385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.059466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.059629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.059709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.059861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.059949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.060135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.060223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.060352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.060435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.060595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.060685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.060840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.061423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.061631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.061808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.061889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.062051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.062147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.062328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.062409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.062581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.062661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.062820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.062902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.063067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] resource.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.063147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.063299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.063385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.063554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.063636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.063789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.063869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.064977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.065120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.065205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.065345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.065434 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] role.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.065600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.065689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] role.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.065838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.065927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.066161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.066251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.066406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.066496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.066643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] token.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.066733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.066880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.066969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.067128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.067218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.067364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.067453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.067620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.067709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.067869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.067958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.068103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.068347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.068435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.068589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.068686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.068835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.068924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.069070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.069159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.069306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.069494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.069583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.069736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.069826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.069989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.070088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.070270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.070371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.070512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.070602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.070775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.070910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.connection = **** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.071952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.072191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.072274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.072455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.072455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.072717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.072717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.072985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.072985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.073297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.073489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.073599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.073686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.073773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.073857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.073999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.074212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.074367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.074520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.074700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] token.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.074787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.074995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.075092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.075158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.075351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.075477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.075563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.075681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.075796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.076001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.076164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:32:04.076291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.076291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.076291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.076673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.076673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.077512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.077753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.078006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.078330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.078581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.078816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.079106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.079399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.connection = **** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.079646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.079899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.080139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.080385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.080661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.080868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.081120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.081347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.081597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.081840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.082087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.082353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.082607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.082848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.083096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.083332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.083606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.083848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:32:04.084023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:32:04.111678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.111927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.112199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.113120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.113356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.113588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.120588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.126335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.155005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.155266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.155393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.155618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.217972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.218175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.269887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.270203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.270829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.271067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.430867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.431165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.431316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.431493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.431695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.431882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.432021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.432149 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.432308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.432528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.432747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.432961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.465931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.466252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.466416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.466572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.466572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.466786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.466916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.467044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.467171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.467361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.467492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.467619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.467746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.467883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.479840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.480065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.496673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.496950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.497091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.497251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.497386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.497518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.497648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.497734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.497917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.498057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.498260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.498477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.498681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.498681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:32:04.515804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.516089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:32:04.516348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.516506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.516678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:32:04.516848 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:32:04.517007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.517156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.517326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.517496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:32:04.517644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.517803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.517952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.518101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.518101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.518101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.527339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.527629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.527909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.528090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.528224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.528384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.528666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.528962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.535869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.536722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.538813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.539104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.539400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.539644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.539775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.539932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.540030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.540215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.540299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.540493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.540587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.540769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.540862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.541047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.541131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.541328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.541410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.541602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.541686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.541880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.542170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.542446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.564907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.565631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.566465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.566760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.567141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.567427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.571478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.571773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.572061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.572145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.572350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.572635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.579108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.579409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.579620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.579870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.585238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.585547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.585661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.585841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.585961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.586230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.586340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.586470 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.586582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.586712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.586887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.586977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.587173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.587256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.587444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.587714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.603335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.603630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.603762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.603991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.607502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.608095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.610799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.611124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.611434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.611779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.615286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.615995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.618391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.619135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.621595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.621896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.623028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.623118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.623196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.623286 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.623366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.623963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.624039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.624124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.624124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.624353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.640482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.640799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.641081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.641349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.641444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.641726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.641998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.642289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.648274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.649024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.649848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.650572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.651409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.651701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.652070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.652347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.654452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.654778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.655066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.655163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.655376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.655466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.655646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.655766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.655931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.656320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.666955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.667245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.667383 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.667522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.667672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.667807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.667952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.668178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.668299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.668502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.668654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.668813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.668919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.669201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.684343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.686672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.687896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.688403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.690854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.691599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.692334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.693312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.693806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.694098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.694539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.695205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.695543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.695998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.701013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.703146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.703581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.705259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.706183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.706588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.706880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.707165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.708748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.709217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.709590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.709990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.715508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.715826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.716119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.718261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.718569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.718851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.723100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.723394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.723682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.723964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.724245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.724523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.724803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.725083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.725359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.725646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.725739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:32:04.725934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.726028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.726223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.726317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.726505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.726610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.726775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.726867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.727048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.727331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.727621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.727892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.728175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.728449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.728716 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.728981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.729249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:32:04.761708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561bd7ad69c0 pid: 88256 (default app) Sep 08 13:32:04.764010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561bd7ad69c0 pid: 88255 (default app) Sep 08 13:32:04.765722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87654430-221d-4da4-a29b-5360066928d8 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:04.766014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87654430-221d-4da4-a29b-5360066928d8 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:04.766324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87654430-221d-4da4-a29b-5360066928d8 None None] PATH_INFO: `/v3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:04.770562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Thu Sep 8 13:32:04 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:12.782046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22ce0a-e84b-4ea4-9e14-f79f65e890f6 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:12.782679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22ce0a-e84b-4ea4-9e14-f79f65e890f6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:12.782679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22ce0a-e84b-4ea4-9e14-f79f65e890f6 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:12.786983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:12.793417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:12.793969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:12.794312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:12.868811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: INFO dbcounter [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] Registered counter for database keystone Sep 08 13:32:12.891416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG oslo_db.sqlalchemy.engines [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88256) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:32:12.897817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writer thread running {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:32:13.082833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG passlib.handlers.bcrypt [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88256) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 13:32:13.083222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG passlib.handlers.bcrypt [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88256) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 13:32:13.144466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:13.146060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:13.287251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 08 13:32:13.787382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22ce0a-e84b-4ea4-9e14-f79f65e890f6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:13.866028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: INFO dbcounter [None req-8c22ce0a-e84b-4ea4-9e14-f79f65e890f6 None None] Registered counter for database keystone Sep 08 13:32:13.886211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG oslo_db.sqlalchemy.engines [None req-8c22ce0a-e84b-4ea4-9e14-f79f65e890f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88255) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:32:13.893337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writer thread running {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:32:14.015114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22ce0a-e84b-4ea4-9e14-f79f65e890f6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:14.019642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91055bd-0b22-45cb-aece-0a3d246f0d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:14.020291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91055bd-0b22-45cb-aece-0a3d246f0d14 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:14.020574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91055bd-0b22-45cb-aece-0a3d246f0d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:14.020848 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91055bd-0b22-45cb-aece-0a3d246f0d14 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:14.030213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91055bd-0b22-45cb-aece-0a3d246f0d14 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:14.083592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d91055bd-0b22-45cb-aece-0a3d246f0d14 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 13:32:14.084142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91055bd-0b22-45cb-aece-0a3d246f0d14 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:14.095330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 8 13:32:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 13:32:14.102901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:14.120167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205200a-5f25-44f6-b762-32692b548d71 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:14.124691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66598f2-0fd4-46e7-8aa4-d21dd309ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:14.125842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66598f2-0fd4-46e7-8aa4-d21dd309ab28 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:14.126128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66598f2-0fd4-46e7-8aa4-d21dd309ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:14.126438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66598f2-0fd4-46e7-8aa4-d21dd309ab28 admin admin] PATH_INFO: `/v3/projects/273637bb45fb4aad868a7d2db9ada393` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:14.128645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66598f2-0fd4-46e7-8aa4-d21dd309ab28 admin admin] RBAC: Authorizing `identity:get_project(project_id=273637bb45fb4aad868a7d2db9ada393)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:14.185228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c66598f2-0fd4-46e7-8aa4-d21dd309ab28 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 13:32:14.185489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66598f2-0fd4-46e7-8aa4-d21dd309ab28 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:14.190972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Thu Sep 8 13:32:14 2022] GET /identity/v3/projects/273637bb45fb4aad868a7d2db9ada393 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:16.386449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afe1f0-0b72-433f-846e-65634e79e37d None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:16.386997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afe1f0-0b72-433f-846e-65634e79e37d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:16.386997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afe1f0-0b72-433f-846e-65634e79e37d None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:16.391400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:16.397630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910013f-ccd4-48b7-9ffa-6ac85ff1c40c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:16.398124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910013f-ccd4-48b7-9ffa-6ac85ff1c40c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:16.398232 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910013f-ccd4-48b7-9ffa-6ac85ff1c40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:16.418185 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-2910013f-ccd4-48b7-9ffa-6ac85ff1c40c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:16.419754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2910013f-ccd4-48b7-9ffa-6ac85ff1c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:16.428387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 08 13:32:16.702312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afe1f0-0b72-433f-846e-65634e79e37d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:16.715269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afe1f0-0b72-433f-846e-65634e79e37d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:16.720133 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27c2b0-bfd8-41ac-ad24-2c3e3b377240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:16.720746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c2b0-bfd8-41ac-ad24-2c3e3b377240 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:16.721033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c2b0-bfd8-41ac-ad24-2c3e3b377240 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:16.721311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c2b0-bfd8-41ac-ad24-2c3e3b377240 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:16.729495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27c2b0-bfd8-41ac-ad24-2c3e3b377240 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:16.731361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27c2b0-bfd8-41ac-ad24-2c3e3b377240 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:16.741603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 8 13:32:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 13:32:16.747461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910013f-ccd4-48b7-9ffa-6ac85ff1c40c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:16.757359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910013f-ccd4-48b7-9ffa-6ac85ff1c40c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:16.761916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9e5a1-3da2-47ad-b8d7-1de0c607ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:16.762674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e5a1-3da2-47ad-b8d7-1de0c607ba61 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:16.762961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e5a1-3da2-47ad-b8d7-1de0c607ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:16.763237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e5a1-3da2-47ad-b8d7-1de0c607ba61 admin admin] PATH_INFO: `/v3/users/4c5c0285558342f3b0682eed94f7ff4b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:16.765177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9e5a1-3da2-47ad-b8d7-1de0c607ba61 admin admin] RBAC: Authorizing `identity:get_user(user_id=4c5c0285558342f3b0682eed94f7ff4b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:16.767132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9e5a1-3da2-47ad-b8d7-1de0c607ba61 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:16.768724 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Thu Sep 8 13:32:16 2022] GET /identity/v3/users/4c5c0285558342f3b0682eed94f7ff4b => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:19.069182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd585b-2de3-443d-8a2b-ab784968c415 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:19.069720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd585b-2de3-443d-8a2b-ab784968c415 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:19.069720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd585b-2de3-443d-8a2b-ab784968c415 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:19.071028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:19.076369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167a322-ef24-4390-bd5a-00e6d9b7eaa1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:19.076845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167a322-ef24-4390-bd5a-00e6d9b7eaa1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:19.077543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167a322-ef24-4390-bd5a-00e6d9b7eaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:19.102797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e167a322-ef24-4390-bd5a-00e6d9b7eaa1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:19.104020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e167a322-ef24-4390-bd5a-00e6d9b7eaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:19.114494 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef68fc-ef7e-48ed-8a22-f7095f484b3b None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:19.114781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef68fc-ef7e-48ed-8a22-f7095f484b3b None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:19.115051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef68fc-ef7e-48ed-8a22-f7095f484b3b None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:19.116393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:32:19.121978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:19.122295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:19.122577 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:19.166436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG passlib.handlers.bcrypt [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88255) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 13:32:19.166838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG passlib.handlers.bcrypt [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88255) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 13:32:19.195931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:19.223690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:19.225414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:19.339687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:19.474720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167a322-ef24-4390-bd5a-00e6d9b7eaa1 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:19.500278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:19.596849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167a322-ef24-4390-bd5a-00e6d9b7eaa1 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:19.610703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:19.632231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4564f-c8ae-4f9f-b55d-7406080b37b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:19.632889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4564f-c8ae-4f9f-b55d-7406080b37b6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:19.633169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4564f-c8ae-4f9f-b55d-7406080b37b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:19.633449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4564f-c8ae-4f9f-b55d-7406080b37b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:19.645580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4564f-c8ae-4f9f-b55d-7406080b37b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-78d4564f-c8ae-4f9f-b55d-7406080b37b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:19.656291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:19.657808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:19.658846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:32:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:19.663927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4564f-c8ae-4f9f-b55d-7406080b37b6 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:19.727350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cce2d-b06a-4c6f-8865-52882e9318fb None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:19.729647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:19.755371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:19.768860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661ec90-8237-41d9-8ca7-381f65014967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:19.768860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661ec90-8237-41d9-8ca7-381f65014967 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:19.769163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661ec90-8237-41d9-8ca7-381f65014967 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:19.769422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661ec90-8237-41d9-8ca7-381f65014967 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:19.774450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4564f-c8ae-4f9f-b55d-7406080b37b6 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:19.819244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:19.819866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:19.820148 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:19.820419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:19.821103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:19.821405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:19.822839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:19.835783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:19.842631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:19.842819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661ec90-8237-41d9-8ca7-381f65014967 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:19.843917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661ec90-8237-41d9-8ca7-381f65014967 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:19.956781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc9840-2c92-4d87-a535-a80fabd0ec58 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:19.960038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:32:19.965972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661ec90-8237-41d9-8ca7-381f65014967 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:19.969743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:19.991547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65297c87-63a9-4a44-aa33-d52a01397372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:19.993097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65297c87-63a9-4a44-aa33-d52a01397372 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:19.993097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65297c87-63a9-4a44-aa33-d52a01397372 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:19.993097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65297c87-63a9-4a44-aa33-d52a01397372 None admin] PATH_INFO: `/v3/users/4c5c0285558342f3b0682eed94f7ff4b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:19.994128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65297c87-63a9-4a44-aa33-d52a01397372 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5c0285558342f3b0682eed94f7ff4b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:19.995630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65297c87-63a9-4a44-aa33-d52a01397372 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:19.997225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:19 2022] GET /identity/v3/users/4c5c0285558342f3b0682eed94f7ff4b => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:20.003471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65297c87-63a9-4a44-aa33-d52a01397372 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:20.021153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:20.082041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661ec90-8237-41d9-8ca7-381f65014967 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:20.119091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65297c87-63a9-4a44-aa33-d52a01397372 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:20.121692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:20.122660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:20.122975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:20.123224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:20.142898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:20.144382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:20.155959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9734ef-68bb-4259-bc65-f038b2249688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:20.156619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9734ef-68bb-4259-bc65-f038b2249688 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:20.156898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9734ef-68bb-4259-bc65-f038b2249688 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:20.157173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9734ef-68bb-4259-bc65-f038b2249688 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:20.158343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9734ef-68bb-4259-bc65-f038b2249688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:20.159795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9734ef-68bb-4259-bc65-f038b2249688 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:20.161455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:32:20.163248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:20.164295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:32:20.165298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:32:20.165298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:32:20.165298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:32:20.165298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:32:20.167212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9734ef-68bb-4259-bc65-f038b2249688 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:20.171749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:20.234959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:20.252671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:20.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9734ef-68bb-4259-bc65-f038b2249688 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:20.284262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d28c-bfc2-4d4e-8f39-c4c90feb9f73 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:20.308468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa5b35-fbb0-497e-8459-dd6b89026a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:20.309129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa5b35-fbb0-497e-8459-dd6b89026a20 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:20.309745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa5b35-fbb0-497e-8459-dd6b89026a20 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:20.310300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa5b35-fbb0-497e-8459-dd6b89026a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:20.312252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa5b35-fbb0-497e-8459-dd6b89026a20 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.domain.id=default, user.id=4c5c0285558342f3b0682eed94f7ff4b) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:20.312580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa5b35-fbb0-497e-8459-dd6b89026a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:20.314051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa5b35-fbb0-497e-8459-dd6b89026a20 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:20.319652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15572e74-9ede-4bea-9d6f-598021072005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:20.320354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15572e74-9ede-4bea-9d6f-598021072005 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:20.320797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15572e74-9ede-4bea-9d6f-598021072005 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:20.320948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 8 13:32:20 2022] GET /identity/v3/role_assignments?user.id=4c5c0285558342f3b0682eed94f7ff4b&scope.domain.id=default&role.id=97f11873550e41dba0cd095afb1dec00 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:20.321073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15572e74-9ede-4bea-9d6f-598021072005 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:20.321743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15572e74-9ede-4bea-9d6f-598021072005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:20.322044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15572e74-9ede-4bea-9d6f-598021072005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:20.323496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15572e74-9ede-4bea-9d6f-598021072005 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:20.347197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:22.726621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257acc2-ec6c-4ab6-a627-7b4cc6e5a852 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:22.727161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257acc2-ec6c-4ab6-a627-7b4cc6e5a852 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:22.727161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257acc2-ec6c-4ab6-a627-7b4cc6e5a852 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:22.728503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:22.740223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b437-6aea-4f15-8a50-fc0bfbf45fdf None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:22.740730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b437-6aea-4f15-8a50-fc0bfbf45fdf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:22.740821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b437-6aea-4f15-8a50-fc0bfbf45fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:22.770885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2e37b437-6aea-4f15-8a50-fc0bfbf45fdf None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:22.772169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2e37b437-6aea-4f15-8a50-fc0bfbf45fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:22.874879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 11/23] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:22.896121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24f412-00e2-4081-a559-ee0175c384fd None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:22.896408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24f412-00e2-4081-a559-ee0175c384fd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:22.896600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24f412-00e2-4081-a559-ee0175c384fd None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:22.898061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:22.916067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819825a-085c-4712-94cf-17cb9af71ffe None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:22.916365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819825a-085c-4712-94cf-17cb9af71ffe None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:22.916633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819825a-085c-4712-94cf-17cb9af71ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:22.939531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-1819825a-085c-4712-94cf-17cb9af71ffe None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:22.940768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1819825a-085c-4712-94cf-17cb9af71ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:22.996756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:23.044294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 12/25] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:23.179191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd26d8f-0051-443a-b8d2-519190a27823 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:23.179374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd26d8f-0051-443a-b8d2-519190a27823 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:23.179649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd26d8f-0051-443a-b8d2-519190a27823 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:23.181088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 14/26] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:23.187807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94b89-f7b2-4cf9-91ee-50f1219a117a None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:23.188156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94b89-f7b2-4cf9-91ee-50f1219a117a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:23.188444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94b89-f7b2-4cf9-91ee-50f1219a117a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:23.215065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9ad94b89-f7b2-4cf9-91ee-50f1219a117a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:23.216321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9ad94b89-f7b2-4cf9-91ee-50f1219a117a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:23.277574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd26d8f-0051-443a-b8d2-519190a27823 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:23.323621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 13/27] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:23.436136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd26d8f-0051-443a-b8d2-519190a27823 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:23.477710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4275b-cc87-43a7-89d9-3fb6b799e8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:23.478399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4275b-cc87-43a7-89d9-3fb6b799e8bb None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:23.478680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4275b-cc87-43a7-89d9-3fb6b799e8bb None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:23.478954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4275b-cc87-43a7-89d9-3fb6b799e8bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:23.494168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4275b-cc87-43a7-89d9-3fb6b799e8bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-3bb4275b-cc87-43a7-89d9-3fb6b799e8bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:23.500060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:23.500921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:23.501684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 15/28] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:32:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:32:23.507089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad94b89-f7b2-4cf9-91ee-50f1219a117a None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:23.618168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4275b-cc87-43a7-89d9-3fb6b799e8bb None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:23.670924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:23.717231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad94b89-f7b2-4cf9-91ee-50f1219a117a None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:23.732193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:23.794783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:23.796125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:23.796394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:23.802160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:23.802160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:23.802160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:23.802160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:23.803928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4275b-cc87-43a7-89d9-3fb6b799e8bb None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:23.831052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 14/29] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:23.831943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:23.882900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:23.883553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:23.888055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:23.888361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:23.919469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:23.921204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:32:23.949480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:23.950465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:32:23.951296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:32:23.951296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:32:23.951296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:23.951296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 16/30] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:23.952070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:23.973317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0acc81-8eea-492b-a7d9-1d7a3c32ad6f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:24.016668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2f347-2969-4f43-8ac1-5313adb5be57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:24.018809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f347-2969-4f43-8ac1-5313adb5be57 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.020696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f347-2969-4f43-8ac1-5313adb5be57 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.021459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f347-2969-4f43-8ac1-5313adb5be57 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.023283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2f347-2969-4f43-8ac1-5313adb5be57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:24.026960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2f347-2969-4f43-8ac1-5313adb5be57 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:24.067901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:24.115751 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acc2da-9450-4b4f-bef7-a2deb96cb3dc None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:24.153014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246cf18-e896-463a-9401-8c301371bc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:24.153721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246cf18-e896-463a-9401-8c301371bc98 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.154011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246cf18-e896-463a-9401-8c301371bc98 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.154362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246cf18-e896-463a-9401-8c301371bc98 None admin] PATH_INFO: `/v3/users/4c5c0285558342f3b0682eed94f7ff4b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.155619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246cf18-e896-463a-9401-8c301371bc98 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5c0285558342f3b0682eed94f7ff4b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:24.157237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246cf18-e896-463a-9401-8c301371bc98 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:24.158886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 17/31] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:23 2022] GET /identity/v3/users/4c5c0285558342f3b0682eed94f7ff4b => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:24.159684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246cf18-e896-463a-9401-8c301371bc98 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:24.266409 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 15/32] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 13:32:23 2022] POST /identity/v3/roles => generated 219 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:24.268286 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb7713-d9c5-42f4-a6d4-c0aefb7f8ef9 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.268811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb7713-d9c5-42f4-a6d4-c0aefb7f8ef9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.269290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb7713-d9c5-42f4-a6d4-c0aefb7f8ef9 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.271263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 16/33] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:24.272392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb7713-d9c5-42f4-a6d4-c0aefb7f8ef9 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:24.298844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246cf18-e896-463a-9401-8c301371bc98 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:24.311305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:24.351791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:24.391031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54a38f-efae-4d91-9e6d-246885139f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:24.391674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54a38f-efae-4d91-9e6d-246885139f83 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.391956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54a38f-efae-4d91-9e6d-246885139f83 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.392235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54a38f-efae-4d91-9e6d-246885139f83 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.392910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54a38f-efae-4d91-9e6d-246885139f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:24.393222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54a38f-efae-4d91-9e6d-246885139f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:24.394695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54a38f-efae-4d91-9e6d-246885139f83 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:24.412489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb7713-d9c5-42f4-a6d4-c0aefb7f8ef9 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:24.439779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 18/34] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:24.445360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6e69c-de74-41ca-86b5-7307bfbc5108 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.446001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6e69c-de74-41ca-86b5-7307bfbc5108 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.447865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6e69c-de74-41ca-86b5-7307bfbc5108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.492582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-31f6e69c-de74-41ca-86b5-7307bfbc5108 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:24.496608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-31f6e69c-de74-41ca-86b5-7307bfbc5108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:24.502442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebe1c4-5223-4e74-a677-5ed8d8d2d559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:24.503509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebe1c4-5223-4e74-a677-5ed8d8d2d559 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.503994 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebe1c4-5223-4e74-a677-5ed8d8d2d559 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.504481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebe1c4-5223-4e74-a677-5ed8d8d2d559 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.506520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ebe1c4-5223-4e74-a677-5ed8d8d2d559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:24.508826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ebe1c4-5223-4e74-a677-5ed8d8d2d559 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:24.510956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 17/35] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:24.512081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebe1c4-5223-4e74-a677-5ed8d8d2d559 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:24.626035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 19/36] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:24.627159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6e69c-de74-41ca-86b5-7307bfbc5108 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:24.645027 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:24.652324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebe1c4-5223-4e74-a677-5ed8d8d2d559 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:24.693488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09141e88-bf65-424d-821f-f6d3395cfcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:24.694437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09141e88-bf65-424d-821f-f6d3395cfcd2 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.694944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09141e88-bf65-424d-821f-f6d3395cfcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.695375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09141e88-bf65-424d-821f-f6d3395cfcd2 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.696242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09141e88-bf65-424d-821f-f6d3395cfcd2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:24.698007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09141e88-bf65-424d-821f-f6d3395cfcd2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:24.754458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6e69c-de74-41ca-86b5-7307bfbc5108 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:24.770972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:24.792831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e31c-4038-4c48-9ccb-a28a96017c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:24.793522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e31c-4038-4c48-9ccb-a28a96017c22 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.793809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e31c-4038-4c48-9ccb-a28a96017c22 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.794093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e31c-4038-4c48-9ccb-a28a96017c22 None admin] PATH_INFO: `/v3/domains/default/users/4c5c0285558342f3b0682eed94f7ff4b/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.796363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945e31c-4038-4c48-9ccb-a28a96017c22 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4c5c0285558342f3b0682eed94f7ff4b, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:24.797906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945e31c-4038-4c48-9ccb-a28a96017c22 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:24.837964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 18/37] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:32:24 2022] POST /identity/v3/projects => generated 301 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:24.944890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae0f1-e84e-4a8a-ab4e-9f5fb06bb87e None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.945811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae0f1-e84e-4a8a-ab4e-9f5fb06bb87e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.946362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae0f1-e84e-4a8a-ab4e-9f5fb06bb87e None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.947996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 19/38] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:24.954676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aede7c0-4640-4012-8ff8-879a8489d24e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:24.955281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aede7c0-4640-4012-8ff8-879a8489d24e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:24.955926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aede7c0-4640-4012-8ff8-879a8489d24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:24.984731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5aede7c0-4640-4012-8ff8-879a8489d24e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:24.986823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5aede7c0-4640-4012-8ff8-879a8489d24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:25.033552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 20/39] 10.222.0.25 () {58 vars in 1361 bytes} [Thu Sep 8 13:32:24 2022] PUT /identity/v3/domains/default/users/4c5c0285558342f3b0682eed94f7ff4b/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:32:25.035005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e31c-4038-4c48-9ccb-a28a96017c22 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:25.072089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 20/40] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 08 13:32:25.073376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398ff1-497d-4b67-a068-0229ae287ae9 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:25.073677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398ff1-497d-4b67-a068-0229ae287ae9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:25.073957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398ff1-497d-4b67-a068-0229ae287ae9 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:25.075989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 21/41] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:25.083892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f17ef-e5fe-473f-9685-011007648075 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:25.084205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f17ef-e5fe-473f-9685-011007648075 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:25.084461 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f17ef-e5fe-473f-9685-011007648075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:25.116079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-eb4f17ef-e5fe-473f-9685-011007648075 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:25.117365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-eb4f17ef-e5fe-473f-9685-011007648075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:25.201220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e31c-4038-4c48-9ccb-a28a96017c22 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:25.224079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 22/42] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:25.259455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095db94-0ec4-4a0f-a49b-6fcc4174e06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:25.260090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095db94-0ec4-4a0f-a49b-6fcc4174e06e None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:25.260376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095db94-0ec4-4a0f-a49b-6fcc4174e06e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:25.260652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095db94-0ec4-4a0f-a49b-6fcc4174e06e None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:25.261342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095db94-0ec4-4a0f-a49b-6fcc4174e06e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:25.263263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095db94-0ec4-4a0f-a49b-6fcc4174e06e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:25.389225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f17ef-e5fe-473f-9685-011007648075 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:25.400717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:25.413314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 21/43] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 13:32:25 2022] POST /identity/v3/roles => generated 225 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:25.520602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f17ef-e5fe-473f-9685-011007648075 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:25.560924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095db94-0ec4-4a0f-a49b-6fcc4174e06e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:25.563413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74dc79-4c52-4456-a6c3-7ca0fe1ba350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:25.564333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74dc79-4c52-4456-a6c3-7ca0fe1ba350 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:25.564590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74dc79-4c52-4456-a6c3-7ca0fe1ba350 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:25.564869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74dc79-4c52-4456-a6c3-7ca0fe1ba350 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:25.565577 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74dc79-4c52-4456-a6c3-7ca0fe1ba350 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:25.567054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74dc79-4c52-4456-a6c3-7ca0fe1ba350 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:25.609918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:25.671001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 23/44] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 13:32:25 2022] POST /identity/v3/roles => generated 223 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:32:25.801776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095db94-0ec4-4a0f-a49b-6fcc4174e06e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:25.841871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f8f3a-e833-4045-96f9-d4607e0a0691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:25.842650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f8f3a-e833-4045-96f9-d4607e0a0691 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:25.842935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f8f3a-e833-4045-96f9-d4607e0a0691 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:25.843262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f8f3a-e833-4045-96f9-d4607e0a0691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:25.844592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f8f3a-e833-4045-96f9-d4607e0a0691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:25.846096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f8f3a-e833-4045-96f9-d4607e0a0691 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:25.848173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 22/45] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:25.856793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74dc79-4c52-4456-a6c3-7ca0fe1ba350 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:25.940037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:25.974824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74dc79-4c52-4456-a6c3-7ca0fe1ba350 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:26.012869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606eb4e4-f119-470e-9fec-c4c65849dd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:26.013499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606eb4e4-f119-470e-9fec-c4c65849dd14 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:26.013789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606eb4e4-f119-470e-9fec-c4c65849dd14 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:26.014106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606eb4e4-f119-470e-9fec-c4c65849dd14 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:26.014831 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606eb4e4-f119-470e-9fec-c4c65849dd14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:26.016309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606eb4e4-f119-470e-9fec-c4c65849dd14 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:26.088973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:32:25 2022] POST /identity/v3/projects => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:27.368601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056ad7-9ca4-471e-9c41-a8e7eca108f2 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:27.369155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056ad7-9ca4-471e-9c41-a8e7eca108f2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:27.369155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056ad7-9ca4-471e-9c41-a8e7eca108f2 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:27.370544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 23/47] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:27.378252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ce53f-cdc3-4d69-bd99-7f1fc8dffa33 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:27.378754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ce53f-cdc3-4d69-bd99-7f1fc8dffa33 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:27.378837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ce53f-cdc3-4d69-bd99-7f1fc8dffa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:27.397286 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-877ce53f-cdc3-4d69-bd99-7f1fc8dffa33 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:27.398526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-877ce53f-cdc3-4d69-bd99-7f1fc8dffa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:27.462825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 25/48] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:27.736456 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056ad7-9ca4-471e-9c41-a8e7eca108f2 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:27.836163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056ad7-9ca4-471e-9c41-a8e7eca108f2 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:27.837682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:27.870847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79aac9-9bd6-4905-b01f-c0f8afb190d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:27.871482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79aac9-9bd6-4905-b01f-c0f8afb190d4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:27.871764 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79aac9-9bd6-4905-b01f-c0f8afb190d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:27.872039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79aac9-9bd6-4905-b01f-c0f8afb190d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:27.881915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79aac9-9bd6-4905-b01f-c0f8afb190d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-3e79aac9-9bd6-4905-b01f-c0f8afb190d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:27.887288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:27.888342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:27.889377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:27.890215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 24/49] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:32:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:27.895345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ce53f-cdc3-4d69-bd99-7f1fc8dffa33 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:27.950599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:28.017738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ce53f-cdc3-4d69-bd99-7f1fc8dffa33 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:28.158786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11e479-e1fe-48d6-8970-03dfb90ffa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:28.159323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11e479-e1fe-48d6-8970-03dfb90ffa11 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:28.159608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11e479-e1fe-48d6-8970-03dfb90ffa11 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:28.159887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11e479-e1fe-48d6-8970-03dfb90ffa11 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:28.160560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11e479-e1fe-48d6-8970-03dfb90ffa11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:28.160867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11e479-e1fe-48d6-8970-03dfb90ffa11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:28.162284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11e479-e1fe-48d6-8970-03dfb90ffa11 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:28.170977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 26/50] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:28.176645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79aac9-9bd6-4905-b01f-c0f8afb190d4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:28.279259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79aac9-9bd6-4905-b01f-c0f8afb190d4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:28.313278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86235fa7-bae7-4f10-b1dd-8525a3ced0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:28.313498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b545d7-da3a-4544-858f-b335f44e6045 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:28.313631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b545d7-da3a-4544-858f-b335f44e6045 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:28.313904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86235fa7-bae7-4f10-b1dd-8525a3ced0d4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:28.313991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b545d7-da3a-4544-858f-b335f44e6045 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:28.314221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86235fa7-bae7-4f10-b1dd-8525a3ced0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:28.314494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86235fa7-bae7-4f10-b1dd-8525a3ced0d4 None admin] PATH_INFO: `/v3/users/4c5c0285558342f3b0682eed94f7ff4b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:28.315495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 27/51] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:32:28.315700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86235fa7-bae7-4f10-b1dd-8525a3ced0d4 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5c0285558342f3b0682eed94f7ff4b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:28.317411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86235fa7-bae7-4f10-b1dd-8525a3ced0d4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:28.322252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 25/52] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:28 2022] GET /identity/v3/users/4c5c0285558342f3b0682eed94f7ff4b => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:28.322401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caccec1-1b4c-437a-b907-e77608a4cd1f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:28.322401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caccec1-1b4c-437a-b907-e77608a4cd1f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:28.322401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caccec1-1b4c-437a-b907-e77608a4cd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:28.325096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86235fa7-bae7-4f10-b1dd-8525a3ced0d4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:28.344530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6caccec1-1b4c-437a-b907-e77608a4cd1f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:28.345782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6caccec1-1b4c-437a-b907-e77608a4cd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:28.358624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:28.417074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 28/53] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:28.429875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86235fa7-bae7-4f10-b1dd-8525a3ced0d4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:28.475955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5f326-0ed2-4639-8ae4-45101993b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:28.476611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f326-0ed2-4639-8ae4-45101993b112 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:28.476892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f326-0ed2-4639-8ae4-45101993b112 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:28.477174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f326-0ed2-4639-8ae4-45101993b112 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:28.478392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5f326-0ed2-4639-8ae4-45101993b112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:28.479872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5f326-0ed2-4639-8ae4-45101993b112 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:28.481478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 26/54] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:28.488121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caccec1-1b4c-437a-b907-e77608a4cd1f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:28.534103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:28.599468 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caccec1-1b4c-437a-b907-e77608a4cd1f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:28.634955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:28.635571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:28.635870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:28.636135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:28.636877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.domain.id=default, user.id=4c5c0285558342f3b0682eed94f7ff4b) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:28.637190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:28.638690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:28.647440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 29/55] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 8 13:32:28 2022] GET /identity/v3/role_assignments?user.id=4c5c0285558342f3b0682eed94f7ff4b&scope.domain.id=default&role.id=97f11873550e41dba0cd095afb1dec00 => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:28.690037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5f326-0ed2-4639-8ae4-45101993b112 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:28.794187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:28.799019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5f326-0ed2-4639-8ae4-45101993b112 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:28.835289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d7b6e-4d5d-41bc-aa37-cc6104e59b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:28.835938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d7b6e-4d5d-41bc-aa37-cc6104e59b68 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:28.836220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d7b6e-4d5d-41bc-aa37-cc6104e59b68 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:28.836496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d7b6e-4d5d-41bc-aa37-cc6104e59b68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:28.837607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d7b6e-4d5d-41bc-aa37-cc6104e59b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:28.839102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d7b6e-4d5d-41bc-aa37-cc6104e59b68 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:28.840955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 27/56] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:28.847087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:28.946949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb909320-9dc1-480b-a338-dc5afbc3156b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:28.954654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:28.981287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be358123-267a-4620-80a3-09318f10b1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:28.981900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358123-267a-4620-80a3-09318f10b1ad None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:28.982215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358123-267a-4620-80a3-09318f10b1ad None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:28.982500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be358123-267a-4620-80a3-09318f10b1ad None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:28.983183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be358123-267a-4620-80a3-09318f10b1ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:28.984646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be358123-267a-4620-80a3-09318f10b1ad None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:29.083137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 30/57] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 8 13:32:28 2022] POST /identity/v3/projects => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:31.251084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53375e-92f6-4555-8d72-2920f77a587b None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:31.251801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53375e-92f6-4555-8d72-2920f77a587b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:31.251801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53375e-92f6-4555-8d72-2920f77a587b None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:31.252987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 28/58] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:31.258225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0bc32-ee94-44f0-9874-16778e4534f5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:31.258617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0bc32-ee94-44f0-9874-16778e4534f5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:31.258799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0bc32-ee94-44f0-9874-16778e4534f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:31.276709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-67f0bc32-ee94-44f0-9874-16778e4534f5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:31.277906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-67f0bc32-ee94-44f0-9874-16778e4534f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:31.342628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 31/59] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:31.615797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53375e-92f6-4555-8d72-2920f77a587b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:31.718350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53375e-92f6-4555-8d72-2920f77a587b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:31.752845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff1ecf-b011-4846-9a16-1e33581ea920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:31.753473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1ecf-b011-4846-9a16-1e33581ea920 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:31.753759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1ecf-b011-4846-9a16-1e33581ea920 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:31.754036 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1ecf-b011-4846-9a16-1e33581ea920 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:31.755183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff1ecf-b011-4846-9a16-1e33581ea920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:31.756647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff1ecf-b011-4846-9a16-1e33581ea920 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:31.758240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 29/60] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:31.763772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0bc32-ee94-44f0-9874-16778e4534f5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:31.847374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:31.859440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0bc32-ee94-44f0-9874-16778e4534f5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:31.892268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabaa30-6170-49b8-9b27-e03f13299bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:31.892822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabaa30-6170-49b8-9b27-e03f13299bb9 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:31.893106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabaa30-6170-49b8-9b27-e03f13299bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:31.893381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabaa30-6170-49b8-9b27-e03f13299bb9 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:31.894044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabaa30-6170-49b8-9b27-e03f13299bb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:31.895536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabaa30-6170-49b8-9b27-e03f13299bb9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:32.007525 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 32/61] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:32:31 2022] POST /identity/v3/users => generated 275 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:35.138115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed27ff-0cd9-4e98-b48c-26737bf56a8f None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.138661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed27ff-0cd9-4e98-b48c-26737bf56a8f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.138739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed27ff-0cd9-4e98-b48c-26737bf56a8f None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.140082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 30/62] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:35.146271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bff1d8-b9d3-4e3d-9ebe-59c9b027807a None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.146685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bff1d8-b9d3-4e3d-9ebe-59c9b027807a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.146876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bff1d8-b9d3-4e3d-9ebe-59c9b027807a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.179179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-88bff1d8-b9d3-4e3d-9ebe-59c9b027807a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:35.180425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-88bff1d8-b9d3-4e3d-9ebe-59c9b027807a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:35.275462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 33/63] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:35.469493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f494543-1a64-42b2-be16-284b6f0f8deb None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.469843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f494543-1a64-42b2-be16-284b6f0f8deb None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.470020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f494543-1a64-42b2-be16-284b6f0f8deb None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.471629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 31/64] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:35.476673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d1abb-c024-4c40-8ce7-848e01519872 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.476971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d1abb-c024-4c40-8ce7-848e01519872 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.477255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d1abb-c024-4c40-8ce7-848e01519872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.503118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e37d1abb-c024-4c40-8ce7-848e01519872 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:35.504327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e37d1abb-c024-4c40-8ce7-848e01519872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:35.591947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 34/65] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:35.717361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f494543-1a64-42b2-be16-284b6f0f8deb None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:35.733854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28ced-5cd9-471d-a4a7-613fff60d094 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.734058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28ced-5cd9-471d-a4a7-613fff60d094 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.734540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28ced-5cd9-471d-a4a7-613fff60d094 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.736026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 35/66] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:35.745756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac9fb5-bd89-486b-838a-397991caae37 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.746250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac9fb5-bd89-486b-838a-397991caae37 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.746677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac9fb5-bd89-486b-838a-397991caae37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.759649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:35.774317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-abac9fb5-bd89-486b-838a-397991caae37 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:35.775627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-abac9fb5-bd89-486b-838a-397991caae37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:35.793864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:35.852371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 36/67] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:35.854195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a500-04c4-4d5f-a351-7398d9d503bc None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.862556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a500-04c4-4d5f-a351-7398d9d503bc None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.862847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a500-04c4-4d5f-a351-7398d9d503bc None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.864382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 37/68] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:35.878152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6800ed-c0e1-49fc-b038-04ed3de0feab None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.878653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6800ed-c0e1-49fc-b038-04ed3de0feab None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.879097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6800ed-c0e1-49fc-b038-04ed3de0feab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.887453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f494543-1a64-42b2-be16-284b6f0f8deb None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:35.913404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5a6800ed-c0e1-49fc-b038-04ed3de0feab None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:35.914640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5a6800ed-c0e1-49fc-b038-04ed3de0feab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:35.929859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dfb0b-cbd5-47ff-ac45-b1954b486fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:35.930544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dfb0b-cbd5-47ff-ac45-b1954b486fc3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:35.930835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dfb0b-cbd5-47ff-ac45-b1954b486fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:35.931114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dfb0b-cbd5-47ff-ac45-b1954b486fc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:35.942814 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dfb0b-cbd5-47ff-ac45-b1954b486fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-da5dfb0b-cbd5-47ff-ac45-b1954b486fc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:35.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:35.948933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:35.949707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 32/69] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:35.950871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dfb0b-cbd5-47ff-ac45-b1954b486fc3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:35.983548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 38/70] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:35.984685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6800ed-c0e1-49fc-b038-04ed3de0feab None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.049991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:36.089762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6800ed-c0e1-49fc-b038-04ed3de0feab None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.130627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:36.140131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dfb0b-cbd5-47ff-ac45-b1954b486fc3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.167635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.168289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.168591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.168876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.169567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:36.169878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:36.172616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:36.191684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 39/71] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:36.197547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.201062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8626b6-561d-4b17-9251-48e55feb3644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.203481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8626b6-561d-4b17-9251-48e55feb3644 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.203936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8626b6-561d-4b17-9251-48e55feb3644 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.204365 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8626b6-561d-4b17-9251-48e55feb3644 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.217806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8626b6-561d-4b17-9251-48e55feb3644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-de8626b6-561d-4b17-9251-48e55feb3644 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:36.223299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:36.224282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:36.225198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:36.227293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 33/72] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:32:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:36.228706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09638b-9dbf-46eb-8f6c-44dab643e577 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.228996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09638b-9dbf-46eb-8f6c-44dab643e577 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.229274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09638b-9dbf-46eb-8f6c-44dab643e577 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.230553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 34/73] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:36.234272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09638b-9dbf-46eb-8f6c-44dab643e577 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.316355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc62472-ea40-44fa-b724-56d6f1e0107f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.339128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09638b-9dbf-46eb-8f6c-44dab643e577 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.341212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:36.349842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eecac9-8621-41fe-a9fc-5f8b6522231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.350508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eecac9-8621-41fe-a9fc-5f8b6522231b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.350796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eecac9-8621-41fe-a9fc-5f8b6522231b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.352728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eecac9-8621-41fe-a9fc-5f8b6522231b None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.366792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eecac9-8621-41fe-a9fc-5f8b6522231b None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:36.368262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eecac9-8621-41fe-a9fc-5f8b6522231b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:36.369903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 40/74] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:36.370704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eecac9-8621-41fe-a9fc-5f8b6522231b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.373456 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db8ed7-c512-4886-baee-2878d0c06a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.374057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db8ed7-c512-4886-baee-2878d0c06a60 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.374380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db8ed7-c512-4886-baee-2878d0c06a60 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.374657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db8ed7-c512-4886-baee-2878d0c06a60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.384807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db8ed7-c512-4886-baee-2878d0c06a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c9db8ed7-c512-4886-baee-2878d0c06a60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:36.390613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:36.391654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:36.392436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 35/75] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:36.393063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3f928-a258-4759-98db-2cd37e5eda95 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.393135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3f928-a258-4759-98db-2cd37e5eda95 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.393704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3f928-a258-4759-98db-2cd37e5eda95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.411267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-cdf3f928-a258-4759-98db-2cd37e5eda95 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:36.412794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cdf3f928-a258-4759-98db-2cd37e5eda95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:36.422041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:36.474729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eecac9-8621-41fe-a9fc-5f8b6522231b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.487929 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 36/76] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:36.488788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3f928-a258-4759-98db-2cd37e5eda95 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.509308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.509926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.510235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.510586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.511261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:36.511567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:36.513030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:36.526365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 41/77] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:36.526365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.570115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:36.600261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3f928-a258-4759-98db-2cd37e5eda95 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.635878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c304c-bc57-41f9-a629-0593971d01cc None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.641396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc57a3-6f31-4adc-90d5-09e0cb07a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.642245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc57a3-6f31-4adc-90d5-09e0cb07a58e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.642354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc57a3-6f31-4adc-90d5-09e0cb07a58e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.642632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc57a3-6f31-4adc-90d5-09e0cb07a58e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.652988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc57a3-6f31-4adc-90d5-09e0cb07a58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-aacc57a3-6f31-4adc-90d5-09e0cb07a58e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:36.658804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:36.659708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:36.660523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 37/78] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:36.665510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc57a3-6f31-4adc-90d5-09e0cb07a58e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.671603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:36.673746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05342a-01b2-4e06-a65b-c7a44ecbbf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.674406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05342a-01b2-4e06-a65b-c7a44ecbbf24 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.674692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05342a-01b2-4e06-a65b-c7a44ecbbf24 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.674968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05342a-01b2-4e06-a65b-c7a44ecbbf24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.748234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05342a-01b2-4e06-a65b-c7a44ecbbf24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:36.749830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05342a-01b2-4e06-a65b-c7a44ecbbf24 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:36.766587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc57a3-6f31-4adc-90d5-09e0cb07a58e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.796290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:36.801073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.801690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.806029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.806029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.806029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:36.806029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:36.806029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:36.815100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 38/79] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:36.815969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.852423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 42/80] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:36.853370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05342a-01b2-4e06-a65b-c7a44ecbbf24 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.882457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:36.915928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebb04b-ad46-4be9-8311-4ebe0dd762f8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.949486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b64cea-888d-4e5f-8f5f-bb2025035259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.950063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b64cea-888d-4e5f-8f5f-bb2025035259 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.950383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b64cea-888d-4e5f-8f5f-bb2025035259 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.950673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b64cea-888d-4e5f-8f5f-bb2025035259 None admin] PATH_INFO: `/v3/users/4c5c0285558342f3b0682eed94f7ff4b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.952360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05342a-01b2-4e06-a65b-c7a44ecbbf24 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:36.952491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b64cea-888d-4e5f-8f5f-bb2025035259 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5c0285558342f3b0682eed94f7ff4b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:36.953854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b64cea-888d-4e5f-8f5f-bb2025035259 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:36.955650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 39/81] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/users/4c5c0285558342f3b0682eed94f7ff4b => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:36.956260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b64cea-888d-4e5f-8f5f-bb2025035259 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:36.987198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:36.987810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:36.988094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:36.988369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:36.989036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:36.989341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:36.990792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:36.999448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 43/82] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:37.000262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.010741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.062216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b64cea-888d-4e5f-8f5f-bb2025035259 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.095641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d905d67-a12d-4608-abf4-8367e2edeab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.096283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d905d67-a12d-4608-abf4-8367e2edeab1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.096570 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d905d67-a12d-4608-abf4-8367e2edeab1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.096850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d905d67-a12d-4608-abf4-8367e2edeab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.097993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d905d67-a12d-4608-abf4-8367e2edeab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:37.098794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345b3a8-ee49-4c03-a5e4-12fe8d58e0fb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.099497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d905d67-a12d-4608-abf4-8367e2edeab1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:37.101115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 40/83] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:37.103061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d905d67-a12d-4608-abf4-8367e2edeab1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.105616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.132963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9925e-abd1-43b1-adaf-65d7ca65cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.133596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9925e-abd1-43b1-adaf-65d7ca65cd45 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.133880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9925e-abd1-43b1-adaf-65d7ca65cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.134177 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9925e-abd1-43b1-adaf-65d7ca65cd45 None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.135308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9925e-abd1-43b1-adaf-65d7ca65cd45 None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:37.136764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9925e-abd1-43b1-adaf-65d7ca65cd45 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:37.138526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 44/84] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:36 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:37.139814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9925e-abd1-43b1-adaf-65d7ca65cd45 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.207729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d905d67-a12d-4608-abf4-8367e2edeab1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.214594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.238408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9925e-abd1-43b1-adaf-65d7ca65cd45 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.241663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ada5a-367d-4557-a2b4-12968dd02490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.242332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ada5a-367d-4557-a2b4-12968dd02490 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.242621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ada5a-367d-4557-a2b4-12968dd02490 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.242920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ada5a-367d-4557-a2b4-12968dd02490 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.244048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ada5a-367d-4557-a2b4-12968dd02490 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:37.245522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ada5a-367d-4557-a2b4-12968dd02490 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:37.247210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 41/85] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:37 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:37.248016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ada5a-367d-4557-a2b4-12968dd02490 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.276369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35429d-78c6-43db-a24a-1c59b1a1309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.276984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35429d-78c6-43db-a24a-1c59b1a1309b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.277271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35429d-78c6-43db-a24a-1c59b1a1309b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.277552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35429d-78c6-43db-a24a-1c59b1a1309b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.315574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.347800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35429d-78c6-43db-a24a-1c59b1a1309b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:37.348002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ada5a-367d-4557-a2b4-12968dd02490 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.349273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35429d-78c6-43db-a24a-1c59b1a1309b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:37.383515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea12dbc-b795-4ede-ac39-648931e0f51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.384085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea12dbc-b795-4ede-ac39-648931e0f51a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.384376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea12dbc-b795-4ede-ac39-648931e0f51a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.384661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea12dbc-b795-4ede-ac39-648931e0f51a None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.385840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea12dbc-b795-4ede-ac39-648931e0f51a None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:37.387340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea12dbc-b795-4ede-ac39-648931e0f51a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:37.388940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 42/86] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:37 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:37.389815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea12dbc-b795-4ede-ac39-648931e0f51a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.423539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.446655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 45/87] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:37.447486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35429d-78c6-43db-a24a-1c59b1a1309b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.491788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea12dbc-b795-4ede-ac39-648931e0f51a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.525104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057f3f0-a83d-4461-97ba-137c00bd2d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.525716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057f3f0-a83d-4461-97ba-137c00bd2d8c None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.526002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057f3f0-a83d-4461-97ba-137c00bd2d8c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.526311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057f3f0-a83d-4461-97ba-137c00bd2d8c None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.526965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057f3f0-a83d-4461-97ba-137c00bd2d8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:37.528515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057f3f0-a83d-4461-97ba-137c00bd2d8c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:37.543859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.545230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35429d-78c6-43db-a24a-1c59b1a1309b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.578823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aff8e-8d91-4b2f-9b8b-f010ef2a01c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.579427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aff8e-8d91-4b2f-9b8b-f010ef2a01c3 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.579711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aff8e-8d91-4b2f-9b8b-f010ef2a01c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.579986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aff8e-8d91-4b2f-9b8b-f010ef2a01c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.624653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.641216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 43/88] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:32:37 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:37.642083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057f3f0-a83d-4461-97ba-137c00bd2d8c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.648278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aff8e-8d91-4b2f-9b8b-f010ef2a01c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:37.649880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aff8e-8d91-4b2f-9b8b-f010ef2a01c3 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:37.753929 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057f3f0-a83d-4461-97ba-137c00bd2d8c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.775343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 46/89] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:37.775343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aff8e-8d91-4b2f-9b8b-f010ef2a01c3 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.796703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.797399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.797688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.797965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.798757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] RBAC: Adding query filter params (role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2, user.id=0a20f1b54d6d470882694c7186cd6fd3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:37.799070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:37.800553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:37.806643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 44/90] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:37 2022] GET /identity/v3/role_assignments?user.id=0a20f1b54d6d470882694c7186cd6fd3&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=1881f748eade41d6aa06dc2097202f09 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:37.807285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:37.882163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.899086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:37.909631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aff8e-8d91-4b2f-9b8b-f010ef2a01c3 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:37.962775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdf532-aa9a-4dc5-81a2-42ec4f94c673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:37.965789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf532-aa9a-4dc5-81a2-42ec4f94c673 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:37.965789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf532-aa9a-4dc5-81a2-42ec4f94c673 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:37.965789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdf532-aa9a-4dc5-81a2-42ec4f94c673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:37.966763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c73cd9-46cc-474c-9b09-1f0bad546b94 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:38.025923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc2166-4e05-4150-bf9c-22ba7efc3091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:38.026610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc2166-4e05-4150-bf9c-22ba7efc3091 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:38.026897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc2166-4e05-4150-bf9c-22ba7efc3091 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:38.027177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc2166-4e05-4150-bf9c-22ba7efc3091 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:38.028449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc2166-4e05-4150-bf9c-22ba7efc3091 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:38.030031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc2166-4e05-4150-bf9c-22ba7efc3091 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:38.031909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 45/91] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:37 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:38.033115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc2166-4e05-4150-bf9c-22ba7efc3091 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:38.063786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdf532-aa9a-4dc5-81a2-42ec4f94c673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:38.065293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdf532-aa9a-4dc5-81a2-42ec4f94c673 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:38.167566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:38.185662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc2166-4e05-4150-bf9c-22ba7efc3091 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:38.197131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 47/92] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:38.198017 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdf532-aa9a-4dc5-81a2-42ec4f94c673 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:38.221497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:38.241577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c1e2d-34d5-4b28-9f05-e746e4ac4018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:38.242263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c1e2d-34d5-4b28-9f05-e746e4ac4018 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:38.242818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c1e2d-34d5-4b28-9f05-e746e4ac4018 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:38.243102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c1e2d-34d5-4b28-9f05-e746e4ac4018 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:38.246957 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c1e2d-34d5-4b28-9f05-e746e4ac4018 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:38.248488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c1e2d-34d5-4b28-9f05-e746e4ac4018 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:38.252520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 46/93] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:38 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:38.254263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c1e2d-34d5-4b28-9f05-e746e4ac4018 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:38.383150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdf532-aa9a-4dc5-81a2-42ec4f94c673 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:38.439935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c1e2d-34d5-4b28-9f05-e746e4ac4018 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:38.448644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:38.449268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:38.449554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:38.449828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:38.450590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2, user.id=4c5c0285558342f3b0682eed94f7ff4b) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:38.450898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:38.452897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:38.463060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 48/94] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:38 2022] GET /identity/v3/role_assignments?user.id=4c5c0285558342f3b0682eed94f7ff4b&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=97f11873550e41dba0cd095afb1dec00 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:38.464135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:38.510162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:38.517101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f20e85-3f28-42e4-a44f-060136134013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:38.517902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f20e85-3f28-42e4-a44f-060136134013 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:38.518209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f20e85-3f28-42e4-a44f-060136134013 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:38.529309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f20e85-3f28-42e4-a44f-060136134013 None admin] PATH_INFO: `/v3/projects/4916304f10a44d759295c5da3237e502` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:38.529309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f20e85-3f28-42e4-a44f-060136134013 None admin] RBAC: Authorizing `identity:get_project(project_id=4916304f10a44d759295c5da3237e502)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:38.529309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f20e85-3f28-42e4-a44f-060136134013 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:38.530969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 47/95] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:38 2022] GET /identity/v3/projects/4916304f10a44d759295c5da3237e502 => generated 312 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:38.536572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f20e85-3f28-42e4-a44f-060136134013 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:38.575168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b7cf2-15aa-4ecf-89ad-ba288d4970cb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:38.612829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd0d13-778e-486a-b523-c4b3ac208c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:38.613804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0d13-778e-486a-b523-c4b3ac208c83 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:38.614342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0d13-778e-486a-b523-c4b3ac208c83 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:38.614850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0d13-778e-486a-b523-c4b3ac208c83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:38.615849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd0d13-778e-486a-b523-c4b3ac208c83 None admin] RBAC: Adding query filter params (role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2, user.id=0a20f1b54d6d470882694c7186cd6fd3) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:38.616380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd0d13-778e-486a-b523-c4b3ac208c83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:38.618054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd0d13-778e-486a-b523-c4b3ac208c83 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:38.625363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 49/96] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:38 2022] GET /identity/v3/role_assignments?user.id=0a20f1b54d6d470882694c7186cd6fd3&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:38.701001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:38.754172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f20e85-3f28-42e4-a44f-060136134013 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:38.809738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b884d-708e-4db7-85d3-e343b9732ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:38.809738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b884d-708e-4db7-85d3-e343b9732ce7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:38.810030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b884d-708e-4db7-85d3-e343b9732ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:38.810098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b884d-708e-4db7-85d3-e343b9732ce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:38.810873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b884d-708e-4db7-85d3-e343b9732ce7 None admin] RBAC: Adding query filter params (role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=4916304f10a44d759295c5da3237e502, user.id=0a20f1b54d6d470882694c7186cd6fd3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:38.811798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b884d-708e-4db7-85d3-e343b9732ce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:38.813370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b884d-708e-4db7-85d3-e343b9732ce7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:38.822885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 48/97] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:38 2022] GET /identity/v3/role_assignments?user.id=0a20f1b54d6d470882694c7186cd6fd3&scope.project.id=4916304f10a44d759295c5da3237e502&role.id=1881f748eade41d6aa06dc2097202f09 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:40.627807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1b64e-7d80-45e0-b86e-27dfea7f3045 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:40.628317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1b64e-7d80-45e0-b86e-27dfea7f3045 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:40.628424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1b64e-7d80-45e0-b86e-27dfea7f3045 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:40.629847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 50/98] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:40.635765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda1e13-e9ee-46a0-ad96-e9b0ceaa1e97 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:40.636627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda1e13-e9ee-46a0-ad96-e9b0ceaa1e97 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:40.637106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda1e13-e9ee-46a0-ad96-e9b0ceaa1e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:40.663651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5bda1e13-e9ee-46a0-ad96-e9b0ceaa1e97 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:40.664891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5bda1e13-e9ee-46a0-ad96-e9b0ceaa1e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:40.756530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 49/99] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:41.017553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a7a11-8b34-4db8-9d96-1646e8292b38 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.017878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a7a11-8b34-4db8-9d96-1646e8292b38 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.018062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a7a11-8b34-4db8-9d96-1646e8292b38 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.019562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 51/100] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:41.024960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c3a0-1533-48b1-8e0f-a303ecafaca3 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.025258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c3a0-1533-48b1-8e0f-a303ecafaca3 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.025542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c3a0-1533-48b1-8e0f-a303ecafaca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.053619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-6783c3a0-1533-48b1-8e0f-a303ecafaca3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:41.054876 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6783c3a0-1533-48b1-8e0f-a303ecafaca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:41.129081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:41.150201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 50/101] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:41.268835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a7a11-8b34-4db8-9d96-1646e8292b38 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:41.270163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf27d64-2db1-4570-bbf8-ced55fef4ea5 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.270533 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf27d64-2db1-4570-bbf8-ced55fef4ea5 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.270822 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf27d64-2db1-4570-bbf8-ced55fef4ea5 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.272337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 51/102] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:32:41.278186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a0331-2c60-4f98-b686-1664039fd769 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.281653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a0331-2c60-4f98-b686-1664039fd769 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.281653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a0331-2c60-4f98-b686-1664039fd769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.299196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-287a0331-2c60-4f98-b686-1664039fd769 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:41.301022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-287a0331-2c60-4f98-b686-1664039fd769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:41.382567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 52/103] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:41.442894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a7a11-8b34-4db8-9d96-1646e8292b38 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:41.452292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:41.495832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f33d5e-2aed-4443-9d58-ef8d361f4e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:41.496530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33d5e-2aed-4443-9d58-ef8d361f4e8a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.496817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33d5e-2aed-4443-9d58-ef8d361f4e8a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.497095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33d5e-2aed-4443-9d58-ef8d361f4e8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.512985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f33d5e-2aed-4443-9d58-ef8d361f4e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-91f33d5e-2aed-4443-9d58-ef8d361f4e8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:41.520122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:41.521062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:32:41.521815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 52/104] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:41.527089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a0331-2c60-4f98-b686-1664039fd769 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:41.688958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a0331-2c60-4f98-b686-1664039fd769 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:41.713787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:41.715792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f33d5e-2aed-4443-9d58-ef8d361f4e8a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:41.744031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60fe4b-2b21-482d-bcf7-048b23de5ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:41.744993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60fe4b-2b21-482d-bcf7-048b23de5ac2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.745274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60fe4b-2b21-482d-bcf7-048b23de5ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.745552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60fe4b-2b21-482d-bcf7-048b23de5ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.746517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60fe4b-2b21-482d-bcf7-048b23de5ac2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:41.746823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60fe4b-2b21-482d-bcf7-048b23de5ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:41.748745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60fe4b-2b21-482d-bcf7-048b23de5ac2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:41.762026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 53/105] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:41.763529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23ada0-17d0-4abf-a683-af3f6299abb6 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.763822 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23ada0-17d0-4abf-a683-af3f6299abb6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.764618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23ada0-17d0-4abf-a683-af3f6299abb6 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.766021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 54/106] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:41.770776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23ada0-17d0-4abf-a683-af3f6299abb6 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:41.908139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f33d5e-2aed-4443-9d58-ef8d361f4e8a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:41.928006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23ada0-17d0-4abf-a683-af3f6299abb6 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:41.947042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71c5de-9bb2-41b0-9a2a-142ed7724d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:41.948040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71c5de-9bb2-41b0-9a2a-142ed7724d13 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.948389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71c5de-9bb2-41b0-9a2a-142ed7724d13 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.948785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71c5de-9bb2-41b0-9a2a-142ed7724d13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.950224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71c5de-9bb2-41b0-9a2a-142ed7724d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:41.952323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71c5de-9bb2-41b0-9a2a-142ed7724d13 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:41.954334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 53/107] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:41.955242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7773fc-0b53-4fc4-9b1d-065f552ca7b6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.955527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7773fc-0b53-4fc4-9b1d-065f552ca7b6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.955803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7773fc-0b53-4fc4-9b1d-065f552ca7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.969407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce6315-f795-40cb-a67b-91325fc1700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:41.970251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce6315-f795-40cb-a67b-91325fc1700e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:41.970648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce6315-f795-40cb-a67b-91325fc1700e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:41.971026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce6315-f795-40cb-a67b-91325fc1700e None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:41.972290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce6315-f795-40cb-a67b-91325fc1700e None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:41.974637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce6315-f795-40cb-a67b-91325fc1700e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:41.976698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 55/108] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:41 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:41.977095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-bc7773fc-0b53-4fc4-9b1d-065f552ca7b6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:41.977468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce6315-f795-40cb-a67b-91325fc1700e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:41.978340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bc7773fc-0b53-4fc4-9b1d-065f552ca7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:42.010001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:42.034685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:42.056308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 54/109] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:42.058546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7773fc-0b53-4fc4-9b1d-065f552ca7b6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:42.109987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce6315-f795-40cb-a67b-91325fc1700e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:42.160978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6257c2-7b50-45c6-8b8c-ec167de79352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:42.162065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6257c2-7b50-45c6-8b8c-ec167de79352 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.162476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6257c2-7b50-45c6-8b8c-ec167de79352 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.163237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6257c2-7b50-45c6-8b8c-ec167de79352 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.178090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6257c2-7b50-45c6-8b8c-ec167de79352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-0f6257c2-7b50-45c6-8b8c-ec167de79352 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:42.185676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:42.187136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:42.187913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 56/110] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:42.188537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6257c2-7b50-45c6-8b8c-ec167de79352 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:42.189112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7773fc-0b53-4fc4-9b1d-065f552ca7b6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:42.230030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9b872-ca7a-4a82-a951-427767f663c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:42.230697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9b872-ca7a-4a82-a951-427767f663c9 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.231041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9b872-ca7a-4a82-a951-427767f663c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.231319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9b872-ca7a-4a82-a951-427767f663c9 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.231980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9b872-ca7a-4a82-a951-427767f663c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:42.233575 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9b872-ca7a-4a82-a951-427767f663c9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:42.298993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6257c2-7b50-45c6-8b8c-ec167de79352 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:42.305765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:42.326162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 55/111] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:32:42 2022] POST /identity/v3/projects => generated 302 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:42.326162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4b208-caab-4f87-9f15-971461eb202c None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.326162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4b208-caab-4f87-9f15-971461eb202c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.326162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4b208-caab-4f87-9f15-971461eb202c None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.326162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 56/112] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:42.326162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4b208-caab-4f87-9f15-971461eb202c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:42.342099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c9a34-7c2a-4005-a4e1-cf55244fc2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:42.342647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9a34-7c2a-4005-a4e1-cf55244fc2c7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.343363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9a34-7c2a-4005-a4e1-cf55244fc2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.343829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9a34-7c2a-4005-a4e1-cf55244fc2c7 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.345740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9a34-7c2a-4005-a4e1-cf55244fc2c7 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:42.347419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9a34-7c2a-4005-a4e1-cf55244fc2c7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:42.349318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 57/113] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:42 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:42.349318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3e27f-ae22-4201-914c-dfb771299d27 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.349318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3e27f-ae22-4201-914c-dfb771299d27 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.349318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3e27f-ae22-4201-914c-dfb771299d27 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.349318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 58/114] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:42.349318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f223c6-ef98-4d23-9b34-689f9d21bbb5 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.354714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f223c6-ef98-4d23-9b34-689f9d21bbb5 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.354714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f223c6-ef98-4d23-9b34-689f9d21bbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.359160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:42.379164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-71f223c6-ef98-4d23-9b34-689f9d21bbb5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:42.380442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-71f223c6-ef98-4d23-9b34-689f9d21bbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:42.459576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4b208-caab-4f87-9f15-971461eb202c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:42.461645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 59/115] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:42.462576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f223c6-ef98-4d23-9b34-689f9d21bbb5 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:42.500034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:42.501006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.501492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.501993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.502906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:42.503320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:42.504747 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:42.515702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 57/116] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:42.516921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:42.609937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f223c6-ef98-4d23-9b34-689f9d21bbb5 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:42.613283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:42.619888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:42.651099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ca978-c622-4c01-ad13-e1fd6a1aa0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:42.651917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ca978-c622-4c01-ad13-e1fd6a1aa0d3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.652182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c7ac9-74c8-4eac-a025-371c48912ff2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:42.652384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ca978-c622-4c01-ad13-e1fd6a1aa0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.652789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ca978-c622-4c01-ad13-e1fd6a1aa0d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.665987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ca978-c622-4c01-ad13-e1fd6a1aa0d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-cf2ca978-c622-4c01-ad13-e1fd6a1aa0d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:42.671350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:42.672236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:42.673522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 60/117] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:32:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:42.674281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa204b-6b89-4be7-876e-b7dc62983360 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.675170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa204b-6b89-4be7-876e-b7dc62983360 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.678258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa204b-6b89-4be7-876e-b7dc62983360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.690318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4663b44-4b41-47ab-8e06-1c54a0a7743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:42.690998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4663b44-4b41-47ab-8e06-1c54a0a7743b None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.691280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4663b44-4b41-47ab-8e06-1c54a0a7743b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.691559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4663b44-4b41-47ab-8e06-1c54a0a7743b None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/users/0a20f1b54d6d470882694c7186cd6fd3/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.693536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4663b44-4b41-47ab-8e06-1c54a0a7743b None admin] RBAC: Authorizing `identity:create_grant(project_id=aa3a59b969bb486b9ddb33d6df49e5e2, user_id=0a20f1b54d6d470882694c7186cd6fd3, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:42.693869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-fbfa204b-6b89-4be7-876e-b7dc62983360 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:42.695084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4663b44-4b41-47ab-8e06-1c54a0a7743b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:42.695246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fbfa204b-6b89-4be7-876e-b7dc62983360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:42.768058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 61/118] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:42.768798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa204b-6b89-4be7-876e-b7dc62983360 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:42.770324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 58/119] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:32:42 2022] PUT /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/users/0a20f1b54d6d470882694c7186cd6fd3/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:32:42.771426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4663b44-4b41-47ab-8e06-1c54a0a7743b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:42.882345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:42.897261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa204b-6b89-4be7-876e-b7dc62983360 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:42.917298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4663b44-4b41-47ab-8e06-1c54a0a7743b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:42.957550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:42.958211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.958502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.958781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:42.959469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:42.959969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:42.961402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:42.964591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:42.973874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 59/120] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:42.974869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:42.990949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c6092-77b8-4555-bc05-d5d15bfbe42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:42.991621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c6092-77b8-4555-bc05-d5d15bfbe42f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:42.991970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c6092-77b8-4555-bc05-d5d15bfbe42f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:42.992250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c6092-77b8-4555-bc05-d5d15bfbe42f None admin] PATH_INFO: `/v3/users/a6087d73196047799351264a5820a57f` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.018307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c6092-77b8-4555-bc05-d5d15bfbe42f None admin] RBAC: Authorizing `identity:get_user(user_id=a6087d73196047799351264a5820a57f)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:43.019917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c6092-77b8-4555-bc05-d5d15bfbe42f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:43.021719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 62/121] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:42 2022] GET /identity/v3/users/a6087d73196047799351264a5820a57f => generated 289 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:43.022542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c6092-77b8-4555-bc05-d5d15bfbe42f None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.100887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c57a-1433-4147-b695-b23ac92adbb0 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:43.167817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84988d93-280b-4e8e-b898-1cd9d1447927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:43.168391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84988d93-280b-4e8e-b898-1cd9d1447927 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:43.168674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84988d93-280b-4e8e-b898-1cd9d1447927 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:43.168951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84988d93-280b-4e8e-b898-1cd9d1447927 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.178522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c6092-77b8-4555-bc05-d5d15bfbe42f None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:43.181287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84988d93-280b-4e8e-b898-1cd9d1447927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-84988d93-280b-4e8e-b898-1cd9d1447927 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:43.187115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:43.194625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 60/122] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:43.196271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84988d93-280b-4e8e-b898-1cd9d1447927 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.253726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430ab25-e1f4-485b-8922-589c9ea606c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:43.255158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430ab25-e1f4-485b-8922-589c9ea606c2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:43.255872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430ab25-e1f4-485b-8922-589c9ea606c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:43.257117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430ab25-e1f4-485b-8922-589c9ea606c2 None admin] PATH_INFO: `/v3/users/4c5c0285558342f3b0682eed94f7ff4b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.258829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430ab25-e1f4-485b-8922-589c9ea606c2 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5c0285558342f3b0682eed94f7ff4b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:43.261990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430ab25-e1f4-485b-8922-589c9ea606c2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:43.262811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:43.268984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 63/123] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:43 2022] GET /identity/v3/users/4c5c0285558342f3b0682eed94f7ff4b => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:43.270174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430ab25-e1f4-485b-8922-589c9ea606c2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.298385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:43.312217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84988d93-280b-4e8e-b898-1cd9d1447927 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:43.352781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777174c-2c4f-4936-8a32-583f04aae866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:43.354440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777174c-2c4f-4936-8a32-583f04aae866 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:43.354912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777174c-2c4f-4936-8a32-583f04aae866 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:43.355394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777174c-2c4f-4936-8a32-583f04aae866 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.402411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430ab25-e1f4-485b-8922-589c9ea606c2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:43.444294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777174c-2c4f-4936-8a32-583f04aae866 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:43.446317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777174c-2c4f-4936-8a32-583f04aae866 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:43.447333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f29233-a160-46d6-86ee-a3cf067c7a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:43.447985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29233-a160-46d6-86ee-a3cf067c7a18 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:43.448276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29233-a160-46d6-86ee-a3cf067c7a18 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:43.448558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29233-a160-46d6-86ee-a3cf067c7a18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.462540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f29233-a160-46d6-86ee-a3cf067c7a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-86f29233-a160-46d6-86ee-a3cf067c7a18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:43.469750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:43.470675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:43.471455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:43.474282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 64/124] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:32:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:43.475274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f29233-a160-46d6-86ee-a3cf067c7a18 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.519906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:43.581848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:43.596997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 61/125] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:43.596997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777174c-2c4f-4936-8a32-583f04aae866 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.609387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f29233-a160-46d6-86ee-a3cf067c7a18 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:43.647542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:43.648367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:43.648845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:43.649295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.650166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:43.650673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:43.652298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:43.666467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 65/126] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:43.667654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.707364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777174c-2c4f-4936-8a32-583f04aae866 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:43.753323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ed01a-66f5-4989-87da-1aef914a2935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:43.753959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed01a-66f5-4989-87da-1aef914a2935 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:43.754277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed01a-66f5-4989-87da-1aef914a2935 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:43.754675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed01a-66f5-4989-87da-1aef914a2935 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.755878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ed01a-66f5-4989-87da-1aef914a2935 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:43.757499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ed01a-66f5-4989-87da-1aef914a2935 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:43.759566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 62/127] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:43 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:43.760125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ed01a-66f5-4989-87da-1aef914a2935 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.785574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a4398-fa5f-4867-a198-386f8956bfc2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:43.796471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:43.824648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:43.825661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:43.826179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:43.826745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.827631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:43.828131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:43.830951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:43.841497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 66/128] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:32:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:43.845670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.850709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:43.890585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ed01a-66f5-4989-87da-1aef914a2935 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:43.934395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544b8f9-00b3-4d59-8d5f-90790424fa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:43.935201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544b8f9-00b3-4d59-8d5f-90790424fa9e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:43.935489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544b8f9-00b3-4d59-8d5f-90790424fa9e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:43.935759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544b8f9-00b3-4d59-8d5f-90790424fa9e None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:43.937472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544b8f9-00b3-4d59-8d5f-90790424fa9e None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:43.938959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544b8f9-00b3-4d59-8d5f-90790424fa9e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:43.940828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 63/129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:43 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:43.942885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544b8f9-00b3-4d59-8d5f-90790424fa9e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:43.991334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69b278-5cb9-406b-a6d1-3caa94a7a294 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:44.037408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a66d26-730d-49a7-b6c5-b8bd48c2b22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:44.038073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a66d26-730d-49a7-b6c5-b8bd48c2b22e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.038386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a66d26-730d-49a7-b6c5-b8bd48c2b22e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.038666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a66d26-730d-49a7-b6c5-b8bd48c2b22e None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.039806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a66d26-730d-49a7-b6c5-b8bd48c2b22e None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:44.041804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a66d26-730d-49a7-b6c5-b8bd48c2b22e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:44.046283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 67/130] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:43 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:44.047404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a66d26-730d-49a7-b6c5-b8bd48c2b22e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:44.053082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:44.064534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544b8f9-00b3-4d59-8d5f-90790424fa9e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:44.113432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15b211-a82d-4488-96eb-2dd18cb5f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:44.114612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15b211-a82d-4488-96eb-2dd18cb5f905 None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.115036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15b211-a82d-4488-96eb-2dd18cb5f905 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.115868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15b211-a82d-4488-96eb-2dd18cb5f905 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/users/4c5c0285558342f3b0682eed94f7ff4b/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.118520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15b211-a82d-4488-96eb-2dd18cb5f905 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa3a59b969bb486b9ddb33d6df49e5e2, user_id=4c5c0285558342f3b0682eed94f7ff4b, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:44.120711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15b211-a82d-4488-96eb-2dd18cb5f905 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:44.128368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:44.165290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a66d26-730d-49a7-b6c5-b8bd48c2b22e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:44.199628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797bd52-fe00-4fce-b7d1-7f246c3786d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:44.200558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797bd52-fe00-4fce-b7d1-7f246c3786d4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.200942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797bd52-fe00-4fce-b7d1-7f246c3786d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.201326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797bd52-fe00-4fce-b7d1-7f246c3786d4 None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.203899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797bd52-fe00-4fce-b7d1-7f246c3786d4 None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:44.222109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797bd52-fe00-4fce-b7d1-7f246c3786d4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:44.222109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 68/131] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:44 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:44.222109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797bd52-fe00-4fce-b7d1-7f246c3786d4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:44.273405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 64/132] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:32:43 2022] PUT /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/users/4c5c0285558342f3b0682eed94f7ff4b/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:32:44.276218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15b211-a82d-4488-96eb-2dd18cb5f905 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:44.428533 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797bd52-fe00-4fce-b7d1-7f246c3786d4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:44.447891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15b211-a82d-4488-96eb-2dd18cb5f905 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:44.491772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d476c-adc1-4bea-be8e-ebc7e38bdecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:44.492757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d476c-adc1-4bea-be8e-ebc7e38bdecd None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.493853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d476c-adc1-4bea-be8e-ebc7e38bdecd None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.494380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d476c-adc1-4bea-be8e-ebc7e38bdecd None admin] PATH_INFO: `/v3/projects/4916304f10a44d759295c5da3237e502` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.496343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d476c-adc1-4bea-be8e-ebc7e38bdecd None admin] RBAC: Authorizing `identity:get_project(project_id=4916304f10a44d759295c5da3237e502)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:44.498082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d476c-adc1-4bea-be8e-ebc7e38bdecd None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:44.502874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 65/133] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:44 2022] GET /identity/v3/projects/4916304f10a44d759295c5da3237e502 => generated 312 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:44.505585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d476c-adc1-4bea-be8e-ebc7e38bdecd None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:44.510903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:44.511793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.512214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.512662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.513557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] RBAC: Adding query filter params (role.id=44854c86be404c76a5703d725444f549, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2, user.id=a6087d73196047799351264a5820a57f) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:44.514045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:44.516429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:44.527516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 69/134] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:44 2022] GET /identity/v3/role_assignments?user.id=a6087d73196047799351264a5820a57f&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=44854c86be404c76a5703d725444f549 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:44.534164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:44.535698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:44.542946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:44.638182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d476c-adc1-4bea-be8e-ebc7e38bdecd None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:44.692984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487b2f3-e521-432e-9f1f-cff2a4e9dfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:44.694272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487b2f3-e521-432e-9f1f-cff2a4e9dfcf None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.694800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487b2f3-e521-432e-9f1f-cff2a4e9dfcf None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.695280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487b2f3-e521-432e-9f1f-cff2a4e9dfcf None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.698710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487b2f3-e521-432e-9f1f-cff2a4e9dfcf None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:44.700568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487b2f3-e521-432e-9f1f-cff2a4e9dfcf None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:44.702707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 66/135] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:44 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:44.708752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378c201-1cba-4798-8f4e-98510cdd00a2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:44.712509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487b2f3-e521-432e-9f1f-cff2a4e9dfcf None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:44.794257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3adf0-6f14-482b-9c5c-f5cfded54640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:44.794257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3adf0-6f14-482b-9c5c-f5cfded54640 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.794257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3adf0-6f14-482b-9c5c-f5cfded54640 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.794257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3adf0-6f14-482b-9c5c-f5cfded54640 None admin] PATH_INFO: `/v3/projects/4916304f10a44d759295c5da3237e502/users/0a20f1b54d6d470882694c7186cd6fd3/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.799214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3adf0-6f14-482b-9c5c-f5cfded54640 None admin] RBAC: Authorizing `identity:create_grant(project_id=4916304f10a44d759295c5da3237e502, user_id=0a20f1b54d6d470882694c7186cd6fd3, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:44.800881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3adf0-6f14-482b-9c5c-f5cfded54640 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:44.851335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:44.861986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487b2f3-e521-432e-9f1f-cff2a4e9dfcf None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:44.901301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372896c-7ad2-4ac3-99b2-0add7a230110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:44.901982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372896c-7ad2-4ac3-99b2-0add7a230110 None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.902293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372896c-7ad2-4ac3-99b2-0add7a230110 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.902573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372896c-7ad2-4ac3-99b2-0add7a230110 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/users/0a20f1b54d6d470882694c7186cd6fd3/roles/4ed4b4b515ca4823abbe45566fe27161` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.905294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 70/136] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:32:44 2022] PUT /identity/v3/projects/4916304f10a44d759295c5da3237e502/users/0a20f1b54d6d470882694c7186cd6fd3/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:32:44.906578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542095-21f9-461e-b028-73cd1a3a32bd None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.906865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542095-21f9-461e-b028-73cd1a3a32bd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.907149 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542095-21f9-461e-b028-73cd1a3a32bd None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.909623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 71/137] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:44.914681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372896c-7ad2-4ac3-99b2-0add7a230110 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa3a59b969bb486b9ddb33d6df49e5e2, user_id=0a20f1b54d6d470882694c7186cd6fd3, role_id=4ed4b4b515ca4823abbe45566fe27161)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:44.916395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372896c-7ad2-4ac3-99b2-0add7a230110 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:44.921118 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035a524-aed0-4327-a62a-5eccc435cd02 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:44.921383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035a524-aed0-4327-a62a-5eccc435cd02 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:44.921609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035a524-aed0-4327-a62a-5eccc435cd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:44.953794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-3035a524-aed0-4327-a62a-5eccc435cd02 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:44.955357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3035a524-aed0-4327-a62a-5eccc435cd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:44.996876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 67/138] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:32:44 2022] PUT /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/users/0a20f1b54d6d470882694c7186cd6fd3/roles/4ed4b4b515ca4823abbe45566fe27161 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:32:45.041543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:45.052276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 72/139] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:45.442670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372896c-7ad2-4ac3-99b2-0add7a230110 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:45.475087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce651b-a055-4104-944b-c030511a0e64 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:45.475410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce651b-a055-4104-944b-c030511a0e64 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:45.475621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce651b-a055-4104-944b-c030511a0e64 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:45.477238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 73/140] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:32:45.488152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18569a-c4fb-4a05-bd28-6c4c82165224 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:45.488429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18569a-c4fb-4a05-bd28-6c4c82165224 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:45.488710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18569a-c4fb-4a05-bd28-6c4c82165224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:45.522479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-4b18569a-c4fb-4a05-bd28-6c4c82165224 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:45.523703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4b18569a-c4fb-4a05-bd28-6c4c82165224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:45.634834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372896c-7ad2-4ac3-99b2-0add7a230110 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:45.699971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 74/141] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:32:45.720525 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73402d-9a02-4c4c-9640-56a8c3698417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:45.721183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73402d-9a02-4c4c-9640-56a8c3698417 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:45.721472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73402d-9a02-4c4c-9640-56a8c3698417 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:45.721755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73402d-9a02-4c4c-9640-56a8c3698417 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:45.723011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73402d-9a02-4c4c-9640-56a8c3698417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:45.725061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73402d-9a02-4c4c-9640-56a8c3698417 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:45.727028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 68/142] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:45.732266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18569a-c4fb-4a05-bd28-6c4c82165224 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:45.871922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18569a-c4fb-4a05-bd28-6c4c82165224 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:45.901828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:45.922397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff520b-f200-44a3-acdc-879cba4a1eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:45.923020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff520b-f200-44a3-acdc-879cba4a1eca None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:45.923300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff520b-f200-44a3-acdc-879cba4a1eca None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:45.923574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff520b-f200-44a3-acdc-879cba4a1eca None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:45.924246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff520b-f200-44a3-acdc-879cba4a1eca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:45.925711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff520b-f200-44a3-acdc-879cba4a1eca None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:46.002564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73402d-9a02-4c4c-9640-56a8c3698417 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:46.028975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 75/143] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:32:45 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:46.038458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:46.174976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73402d-9a02-4c4c-9640-56a8c3698417 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:46.224027 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba957c4-4043-4568-b8cd-fc465e03a161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:46.224692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba957c4-4043-4568-b8cd-fc465e03a161 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:46.224982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba957c4-4043-4568-b8cd-fc465e03a161 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:46.225265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba957c4-4043-4568-b8cd-fc465e03a161 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:46.243711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba957c4-4043-4568-b8cd-fc465e03a161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-fba957c4-4043-4568-b8cd-fc465e03a161 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:46.249417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:46.250265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:32:46.251032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 69/144] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:32:46.259388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff520b-f200-44a3-acdc-879cba4a1eca None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:46.391683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff520b-f200-44a3-acdc-879cba4a1eca None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:46.447890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:46.448534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:46.448815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:46.449095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:46.449788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] RBAC: Adding query filter params (name=member) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:46.450095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:46.451536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:46.470103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 76/145] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:46.475876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba957c4-4043-4568-b8cd-fc465e03a161 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:46.601836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:46.632990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba957c4-4043-4568-b8cd-fc465e03a161 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:46.678037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3550a-e2f3-4fa1-a59e-ff8b2d9c05f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:46.678756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3550a-e2f3-4fa1-a59e-ff8b2d9c05f4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:46.679049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3550a-e2f3-4fa1-a59e-ff8b2d9c05f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:46.679333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3550a-e2f3-4fa1-a59e-ff8b2d9c05f4 None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:46.680491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3550a-e2f3-4fa1-a59e-ff8b2d9c05f4 None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:46.681973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3550a-e2f3-4fa1-a59e-ff8b2d9c05f4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:46.683615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 70/146] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:46 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:46.690083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:46.761565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:46.830900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d31c6f-8ccb-4c5c-a6df-91b5ae1013bb None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:46.884289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a63df-673f-48dd-bcdc-0dbc38081bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:46.885656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a63df-673f-48dd-bcdc-0dbc38081bd1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:46.886131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a63df-673f-48dd-bcdc-0dbc38081bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:46.886636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a63df-673f-48dd-bcdc-0dbc38081bd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:46.964193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a63df-673f-48dd-bcdc-0dbc38081bd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:46.966267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a63df-673f-48dd-bcdc-0dbc38081bd1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:47.020130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:47.110020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 77/147] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:47.115312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3550a-e2f3-4fa1-a59e-ff8b2d9c05f4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:47.247216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3550a-e2f3-4fa1-a59e-ff8b2d9c05f4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:47.298928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674fa4c-a43c-4fdb-8f40-74691b28652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:47.299756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674fa4c-a43c-4fdb-8f40-74691b28652c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:47.300052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674fa4c-a43c-4fdb-8f40-74691b28652c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:47.300332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674fa4c-a43c-4fdb-8f40-74691b28652c None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:47.301563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674fa4c-a43c-4fdb-8f40-74691b28652c None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:47.303088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674fa4c-a43c-4fdb-8f40-74691b28652c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:47.304918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 71/148] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:47 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:47.315279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a63df-673f-48dd-bcdc-0dbc38081bd1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:47.510050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a63df-673f-48dd-bcdc-0dbc38081bd1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:47.571825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e3f5-53f8-40f7-ac68-409b40457d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:47.572457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e3f5-53f8-40f7-ac68-409b40457d9a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:47.572739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e3f5-53f8-40f7-ac68-409b40457d9a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:47.573012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e3f5-53f8-40f7-ac68-409b40457d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:47.573762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e3f5-53f8-40f7-ac68-409b40457d9a None admin] RBAC: Adding query filter params (role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2, user.id=0a20f1b54d6d470882694c7186cd6fd3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:47.574070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e3f5-53f8-40f7-ac68-409b40457d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:47.575677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e3f5-53f8-40f7-ac68-409b40457d9a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:47.583235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 78/149] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:47 2022] GET /identity/v3/role_assignments?user.id=0a20f1b54d6d470882694c7186cd6fd3&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=1881f748eade41d6aa06dc2097202f09 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:47.756019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253c8e-53b5-44d9-9071-3fe1dab25e12 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:47.756358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253c8e-53b5-44d9-9071-3fe1dab25e12 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:47.756612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253c8e-53b5-44d9-9071-3fe1dab25e12 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:47.758211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 72/150] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:47.767037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e4c8-7087-406f-ae0f-16d0e0481a57 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:47.767212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e4c8-7087-406f-ae0f-16d0e0481a57 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:47.767489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e4c8-7087-406f-ae0f-16d0e0481a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:47.790850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:47.794526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-b629e4c8-7087-406f-ae0f-16d0e0481a57 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:47.796460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b629e4c8-7087-406f-ae0f-16d0e0481a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:47.895964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 79/151] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:48.016757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0f97-6f13-4550-92e4-27ca78c45477 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:48.017165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0f97-6f13-4550-92e4-27ca78c45477 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:48.017344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0f97-6f13-4550-92e4-27ca78c45477 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:48.019006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 73/152] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:48.024300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b99307-5e3f-41c1-a956-62da3ddfe95b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:48.024592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b99307-5e3f-41c1-a956-62da3ddfe95b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:48.024874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b99307-5e3f-41c1-a956-62da3ddfe95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:48.048172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-99b99307-5e3f-41c1-a956-62da3ddfe95b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:48.049389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-99b99307-5e3f-41c1-a956-62da3ddfe95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:48.140587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 80/153] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:48.487433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0f97-6f13-4550-92e4-27ca78c45477 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:48.504795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:48.620698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0f97-6f13-4550-92e4-27ca78c45477 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:48.665531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075fd28-cd9b-4ac3-8c02-a013475987ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:48.666223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fd28-cd9b-4ac3-8c02-a013475987ee None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:48.666509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fd28-cd9b-4ac3-8c02-a013475987ee None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:48.666784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fd28-cd9b-4ac3-8c02-a013475987ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:48.679795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075fd28-cd9b-4ac3-8c02-a013475987ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-d075fd28-cd9b-4ac3-8c02-a013475987ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:48.689042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:48.690035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:32:48.690911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 74/154] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:48.699306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b99307-5e3f-41c1-a956-62da3ddfe95b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:48.705238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3744f-96c9-4389-8d19-8a7f1eee47e4 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:48.705566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3744f-96c9-4389-8d19-8a7f1eee47e4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:48.705801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3744f-96c9-4389-8d19-8a7f1eee47e4 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:48.707473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 75/155] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:48.713189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccc31c-dd1b-4b64-baf2-25a8a14cfb52 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:48.713478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccc31c-dd1b-4b64-baf2-25a8a14cfb52 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:48.713749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccc31c-dd1b-4b64-baf2-25a8a14cfb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:48.733981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-efccc31c-dd1b-4b64-baf2-25a8a14cfb52 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:48.735237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-efccc31c-dd1b-4b64-baf2-25a8a14cfb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:48.789122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:48.825347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b99307-5e3f-41c1-a956-62da3ddfe95b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:48.863719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6dbe8-0557-4053-8cb2-89e3209609ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:48.864560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6dbe8-0557-4053-8cb2-89e3209609ee None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:48.865087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6dbe8-0557-4053-8cb2-89e3209609ee None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:48.865370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6dbe8-0557-4053-8cb2-89e3209609ee None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:48.866056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6dbe8-0557-4053-8cb2-89e3209609ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:48.866393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6dbe8-0557-4053-8cb2-89e3209609ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:48.867806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6dbe8-0557-4053-8cb2-89e3209609ee None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:48.883382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 81/156] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:48.884446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90618-3234-4913-b973-45528a63358d None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:48.884734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90618-3234-4913-b973-45528a63358d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:48.885009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90618-3234-4913-b973-45528a63358d None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:48.886360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 82/157] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:48.886935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed90618-3234-4913-b973-45528a63358d None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:48.899750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 76/158] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:48.901049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb24f-6364-4912-be44-989ad2e2ca68 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:48.901339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb24f-6364-4912-be44-989ad2e2ca68 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:48.901617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb24f-6364-4912-be44-989ad2e2ca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:48.925088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:48.942387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f5ccb24f-6364-4912-be44-989ad2e2ca68 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:48.943609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f5ccb24f-6364-4912-be44-989ad2e2ca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:49.107495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 77/159] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:49.108567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccb24f-6364-4912-be44-989ad2e2ca68 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:49.147126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed90618-3234-4913-b973-45528a63358d None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:49.195507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:49.209650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1053564-363d-48f0-a086-da3e93cb6b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:49.218495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053564-363d-48f0-a086-da3e93cb6b3b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:49.218495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053564-363d-48f0-a086-da3e93cb6b3b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:49.218495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053564-363d-48f0-a086-da3e93cb6b3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:49.229728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1053564-363d-48f0-a086-da3e93cb6b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-e1053564-363d-48f0-a086-da3e93cb6b3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:49.238930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:49.239825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:49.240688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 83/160] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:32:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:49.263513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1053564-363d-48f0-a086-da3e93cb6b3b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:49.267993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccb24f-6364-4912-be44-989ad2e2ca68 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:49.318880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4434b-4c29-42e6-8af2-89ff8a9ee670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:49.319538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4434b-4c29-42e6-8af2-89ff8a9ee670 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:49.319819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4434b-4c29-42e6-8af2-89ff8a9ee670 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:49.320096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4434b-4c29-42e6-8af2-89ff8a9ee670 None admin] PATH_INFO: `/v3/users/a6087d73196047799351264a5820a57f` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:49.321285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4434b-4c29-42e6-8af2-89ff8a9ee670 None admin] RBAC: Authorizing `identity:get_user(user_id=a6087d73196047799351264a5820a57f)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:49.322785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4434b-4c29-42e6-8af2-89ff8a9ee670 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:49.324388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 78/161] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:49 2022] GET /identity/v3/users/a6087d73196047799351264a5820a57f => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:49.330320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4434b-4c29-42e6-8af2-89ff8a9ee670 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:49.336171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:49.393110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1053564-363d-48f0-a086-da3e93cb6b3b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:49.432034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:49.433112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:49.433667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:49.434168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:49.435090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:49.435576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:49.437905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:49.447182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 84/162] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:49.448286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:49.507141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:49.544311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4434b-4c29-42e6-8af2-89ff8a9ee670 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:49.565675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27ebf7-258c-4999-9053-327793aa0217 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:49.609427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250d802-04eb-4cbe-bace-fc79e79fb632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:49.610561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250d802-04eb-4cbe-bace-fc79e79fb632 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:49.611059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250d802-04eb-4cbe-bace-fc79e79fb632 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:49.611516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250d802-04eb-4cbe-bace-fc79e79fb632 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:49.631248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250d802-04eb-4cbe-bace-fc79e79fb632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-a250d802-04eb-4cbe-bace-fc79e79fb632 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:49.637537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:49.638467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:49.639348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:49.642420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 85/163] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:32:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:49.644146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250d802-04eb-4cbe-bace-fc79e79fb632 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:49.694872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7180a-f0eb-4821-92e1-d2f03a6ce1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:49.695537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7180a-f0eb-4821-92e1-d2f03a6ce1a8 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:49.695823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7180a-f0eb-4821-92e1-d2f03a6ce1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:49.696099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7180a-f0eb-4821-92e1-d2f03a6ce1a8 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:49.702866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7180a-f0eb-4821-92e1-d2f03a6ce1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:49.704418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7180a-f0eb-4821-92e1-d2f03a6ce1a8 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:49.706041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 79/164] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:49 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:49.706847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7180a-f0eb-4821-92e1-d2f03a6ce1a8 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:49.760384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250d802-04eb-4cbe-bace-fc79e79fb632 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:49.773850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:49.800882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d528-bcc1-411f-9855-77a57091c64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:49.806636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d528-bcc1-411f-9855-77a57091c64b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:49.806636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d528-bcc1-411f-9855-77a57091c64b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:49.806636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6d528-bcc1-411f-9855-77a57091c64b None admin] PATH_INFO: `/v3/users/4c5c0285558342f3b0682eed94f7ff4b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:49.806636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6d528-bcc1-411f-9855-77a57091c64b None admin] RBAC: Authorizing `identity:get_user(user_id=4c5c0285558342f3b0682eed94f7ff4b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:49.806636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6d528-bcc1-411f-9855-77a57091c64b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:49.807874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 86/165] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:49 2022] GET /identity/v3/users/4c5c0285558342f3b0682eed94f7ff4b => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:49.808815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d528-bcc1-411f-9855-77a57091c64b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:49.921435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6d528-bcc1-411f-9855-77a57091c64b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:49.958667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:49.959594 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:49.960046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:49.960508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:49.961519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:49.962003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:49.963673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:49.973873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 87/166] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:32:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:49.975145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:49.976547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:50.027966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:50.085244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42518-b7f1-4353-b608-ec6a17ab5a1b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:50.088402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7180a-f0eb-4821-92e1-d2f03a6ce1a8 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:50.122586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39de81-2168-44a2-a80b-dc0489b9eb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:50.124575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39de81-2168-44a2-a80b-dc0489b9eb75 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:50.125042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39de81-2168-44a2-a80b-dc0489b9eb75 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:50.125539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39de81-2168-44a2-a80b-dc0489b9eb75 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/users/a6087d73196047799351264a5820a57f/roles/44854c86be404c76a5703d725444f549` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:50.127866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39de81-2168-44a2-a80b-dc0489b9eb75 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa3a59b969bb486b9ddb33d6df49e5e2, user_id=a6087d73196047799351264a5820a57f, role_id=44854c86be404c76a5703d725444f549)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:50.129478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39de81-2168-44a2-a80b-dc0489b9eb75 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:50.156840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98591f2-50be-4620-9f0c-90b1e203e02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:50.157784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98591f2-50be-4620-9f0c-90b1e203e02b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:50.158267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98591f2-50be-4620-9f0c-90b1e203e02b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:50.158766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98591f2-50be-4620-9f0c-90b1e203e02b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:50.170440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98591f2-50be-4620-9f0c-90b1e203e02b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:50.179999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 88/167] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:32:49 2022] PUT /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/users/a6087d73196047799351264a5820a57f/roles/44854c86be404c76a5703d725444f549 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:32:50.181468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39de81-2168-44a2-a80b-dc0489b9eb75 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-a98591f2-50be-4620-9f0c-90b1e203e02b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:50.194178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:50.195297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:50.196095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:32:50.200125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 80/168] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 494 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:50.203685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98591f2-50be-4620-9f0c-90b1e203e02b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:50.418093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:50.422270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39de81-2168-44a2-a80b-dc0489b9eb75 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:50.470820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98591f2-50be-4620-9f0c-90b1e203e02b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:50.492715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321591e7-5814-435c-8de9-2cdb87dd2204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:50.493336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321591e7-5814-435c-8de9-2cdb87dd2204 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:50.493617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321591e7-5814-435c-8de9-2cdb87dd2204 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:50.493888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321591e7-5814-435c-8de9-2cdb87dd2204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:50.506667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:50.511837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dfe6d-46f9-4b5c-b805-a06c670e83fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:50.512508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfe6d-46f9-4b5c-b805-a06c670e83fa None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:50.513240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfe6d-46f9-4b5c-b805-a06c670e83fa None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:50.513532 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfe6d-46f9-4b5c-b805-a06c670e83fa None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:50.515956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dfe6d-46f9-4b5c-b805-a06c670e83fa None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:50.517693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dfe6d-46f9-4b5c-b805-a06c670e83fa None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:50.519514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 81/169] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:50 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:50.520297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dfe6d-46f9-4b5c-b805-a06c670e83fa None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:50.633955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321591e7-5814-435c-8de9-2cdb87dd2204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:50.635481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321591e7-5814-435c-8de9-2cdb87dd2204 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:50.644649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dfe6d-46f9-4b5c-b805-a06c670e83fa None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:50.714128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ee0dc-4089-4041-9168-6c73c712b3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:50.714789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ee0dc-4089-4041-9168-6c73c712b3d6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:50.715073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ee0dc-4089-4041-9168-6c73c712b3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:50.715349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ee0dc-4089-4041-9168-6c73c712b3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:50.716292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ee0dc-4089-4041-9168-6c73c712b3d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:50.716604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ee0dc-4089-4041-9168-6c73c712b3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:50.718613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ee0dc-4089-4041-9168-6c73c712b3d6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:50.734309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 82/170] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:50.735507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c8f2c-5e34-45ed-af30-d44b4abb5763 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:50.735798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c8f2c-5e34-45ed-af30-d44b4abb5763 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:50.736080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c8f2c-5e34-45ed-af30-d44b4abb5763 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:50.737440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 83/171] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:50.737990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c8f2c-5e34-45ed-af30-d44b4abb5763 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:50.846444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:50.857909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:50.864381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 89/172] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:50.865600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d915c30-9ed9-4c4a-8c1a-2b0949b55683 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:50.865928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d915c30-9ed9-4c4a-8c1a-2b0949b55683 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:50.866234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d915c30-9ed9-4c4a-8c1a-2b0949b55683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:50.887218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0d915c30-9ed9-4c4a-8c1a-2b0949b55683 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:50.890228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0d915c30-9ed9-4c4a-8c1a-2b0949b55683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:50.937334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c8f2c-5e34-45ed-af30-d44b4abb5763 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:50.975695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505ddca-fa88-4f19-9cd2-5ae6e74714c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:50.976780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ddca-fa88-4f19-9cd2-5ae6e74714c1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:50.977267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ddca-fa88-4f19-9cd2-5ae6e74714c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:50.977755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ddca-fa88-4f19-9cd2-5ae6e74714c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:50.980952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 90/173] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:50.982501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d915c30-9ed9-4c4a-8c1a-2b0949b55683 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:51.051607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505ddca-fa88-4f19-9cd2-5ae6e74714c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:51.053621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505ddca-fa88-4f19-9cd2-5ae6e74714c1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:51.089736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d915c30-9ed9-4c4a-8c1a-2b0949b55683 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:51.116572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:51.121961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:51.128282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382741d5-ea6b-4adb-b22b-827d5da7c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:51.128930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382741d5-ea6b-4adb-b22b-827d5da7c479 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.129213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382741d5-ea6b-4adb-b22b-827d5da7c479 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.130308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382741d5-ea6b-4adb-b22b-827d5da7c479 None admin] PATH_INFO: `/v3/users/0a20f1b54d6d470882694c7186cd6fd3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.130669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382741d5-ea6b-4adb-b22b-827d5da7c479 None admin] RBAC: Authorizing `identity:get_user(user_id=0a20f1b54d6d470882694c7186cd6fd3)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:51.132126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382741d5-ea6b-4adb-b22b-827d5da7c479 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:51.133866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 91/174] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:50 2022] GET /identity/v3/users/0a20f1b54d6d470882694c7186cd6fd3 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:51.134666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382741d5-ea6b-4adb-b22b-827d5da7c479 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:51.163587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 84/175] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:51.165481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4e70f-6e05-43ae-a27d-198bb0f6ebc4 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.165979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4e70f-6e05-43ae-a27d-198bb0f6ebc4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.166469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4e70f-6e05-43ae-a27d-198bb0f6ebc4 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.167994 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 85/176] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:51.173352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9ca4-79c4-494d-95a5-7387aa40dbe8 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.174025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9ca4-79c4-494d-95a5-7387aa40dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.174492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9ca4-79c4-494d-95a5-7387aa40dbe8 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.176052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 86/177] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:51.176967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b9ca4-79c4-494d-95a5-7387aa40dbe8 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:51.250476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382741d5-ea6b-4adb-b22b-827d5da7c479 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:51.288384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c36885-720e-4bed-8128-dd919300e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:51.289132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c36885-720e-4bed-8128-dd919300e10b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.289416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c36885-720e-4bed-8128-dd919300e10b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.289695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c36885-720e-4bed-8128-dd919300e10b None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.290848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c36885-720e-4bed-8128-dd919300e10b None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:51.292564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c36885-720e-4bed-8128-dd919300e10b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:51.294605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 92/178] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:51 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:51.295323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c36885-720e-4bed-8128-dd919300e10b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:51.302373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b9ca4-79c4-494d-95a5-7387aa40dbe8 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:51.339807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe1503-c547-47b2-ae7c-05e7a035eb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:51.340446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe1503-c547-47b2-ae7c-05e7a035eb7d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.340706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe1503-c547-47b2-ae7c-05e7a035eb7d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.340990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe1503-c547-47b2-ae7c-05e7a035eb7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.367611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:51.371216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:51.407976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c36885-720e-4bed-8128-dd919300e10b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:51.413817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe1503-c547-47b2-ae7c-05e7a035eb7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:51.415448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe1503-c547-47b2-ae7c-05e7a035eb7d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:51.441186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c3d5d-13b7-4025-bd86-60144b8c1fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:51.441821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c3d5d-13b7-4025-bd86-60144b8c1fd6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.442108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c3d5d-13b7-4025-bd86-60144b8c1fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.442412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c3d5d-13b7-4025-bd86-60144b8c1fd6 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.443537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c3d5d-13b7-4025-bd86-60144b8c1fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:51.444993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c3d5d-13b7-4025-bd86-60144b8c1fd6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:51.446825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 93/179] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:51 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:51.447761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7178a-2954-4d26-9874-e6a625ed98d9 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.448048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7178a-2954-4d26-9874-e6a625ed98d9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.448328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7178a-2954-4d26-9874-e6a625ed98d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.466874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e3e7178a-2954-4d26-9874-e6a625ed98d9 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:51.468665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e3e7178a-2954-4d26-9874-e6a625ed98d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:51.533463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 87/180] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:51.535116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9300d3-ff9e-4e6f-82e2-b02efe0da1f0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.535574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9300d3-ff9e-4e6f-82e2-b02efe0da1f0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.536017 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9300d3-ff9e-4e6f-82e2-b02efe0da1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.550174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 94/181] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:51.550174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7178a-2954-4d26-9874-e6a625ed98d9 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:51.564742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9d9300d3-ff9e-4e6f-82e2-b02efe0da1f0 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:51.566185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9d9300d3-ff9e-4e6f-82e2-b02efe0da1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:51.626586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:51.641069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:51.652069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 88/182] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:51.658975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9300d3-ff9e-4e6f-82e2-b02efe0da1f0 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:51.694255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7178a-2954-4d26-9874-e6a625ed98d9 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:51.766786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239fac3-9233-4867-bd5a-bf00343d6148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:51.767805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239fac3-9233-4867-bd5a-bf00343d6148 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.768301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239fac3-9233-4867-bd5a-bf00343d6148 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.768779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239fac3-9233-4867-bd5a-bf00343d6148 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.788056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239fac3-9233-4867-bd5a-bf00343d6148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-9239fac3-9233-4867-bd5a-bf00343d6148 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:51.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:51.800173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:51.800949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:51.803411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 95/183] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:51.804285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239fac3-9233-4867-bd5a-bf00343d6148 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:51.806018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9300d3-ff9e-4e6f-82e2-b02efe0da1f0 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:51.847034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:51.847721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:51.848004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:51.848540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:51.850450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2, user.id=4c5c0285558342f3b0682eed94f7ff4b) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:51.850773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:51.853136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:51.866515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 89/184] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:51 2022] GET /identity/v3/role_assignments?user.id=4c5c0285558342f3b0682eed94f7ff4b&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=97f11873550e41dba0cd095afb1dec00 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:51.867346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:51.942387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:51.984408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239fac3-9233-4867-bd5a-bf00343d6148 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:51.996212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaeca0-806f-454b-bf98-82760777e0db None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.029377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:52.034839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.035112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1929e-3b04-4bcc-a64a-1dc45ae1a30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.035766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1929e-3b04-4bcc-a64a-1dc45ae1a30d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.036056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1929e-3b04-4bcc-a64a-1dc45ae1a30d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.036335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1929e-3b04-4bcc-a64a-1dc45ae1a30d None admin] PATH_INFO: `/v3/projects/4916304f10a44d759295c5da3237e502` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.036592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.037057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.037514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.037614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1929e-3b04-4bcc-a64a-1dc45ae1a30d None admin] RBAC: Authorizing `identity:get_project(project_id=4916304f10a44d759295c5da3237e502)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.038594 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] RBAC: Adding query filter params (role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2, user.id=0a20f1b54d6d470882694c7186cd6fd3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:52.039045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1929e-3b04-4bcc-a64a-1dc45ae1a30d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:52.039313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.041004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 90/185] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:51 2022] GET /identity/v3/projects/4916304f10a44d759295c5da3237e502 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:52.046111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:52.046282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1929e-3b04-4bcc-a64a-1dc45ae1a30d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.049002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 96/186] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:51 2022] GET /identity/v3/role_assignments?user.id=0a20f1b54d6d470882694c7186cd6fd3&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:52.055970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.165340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1929e-3b04-4bcc-a64a-1dc45ae1a30d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.192781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:52.205101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.206685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.206685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.206685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.207001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:52.207279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.208692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:52.219039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 91/187] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:52.219906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.225641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156b14b-f25b-46e1-8d90-e9ca0b34deef None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.268653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae620cc-1e4d-484b-a872-0e0bd351c0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.269320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae620cc-1e4d-484b-a872-0e0bd351c0ba None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.269572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae620cc-1e4d-484b-a872-0e0bd351c0ba None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.269841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae620cc-1e4d-484b-a872-0e0bd351c0ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.284093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae620cc-1e4d-484b-a872-0e0bd351c0ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-5ae620cc-1e4d-484b-a872-0e0bd351c0ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:52.290454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:52.291410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:52.292245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 97/188] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:32:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:52.292885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae620cc-1e4d-484b-a872-0e0bd351c0ba None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.346897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c99bb-a7a7-4775-ab3b-b4eb7c32744a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.360515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:52.387020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0230b94-2048-43a5-a7f7-27a1642aef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.387790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0230b94-2048-43a5-a7f7-27a1642aef03 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.388147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0230b94-2048-43a5-a7f7-27a1642aef03 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.388497 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0230b94-2048-43a5-a7f7-27a1642aef03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.389781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0230b94-2048-43a5-a7f7-27a1642aef03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.392212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0230b94-2048-43a5-a7f7-27a1642aef03 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:52.393906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 92/189] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:52.394744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0230b94-2048-43a5-a7f7-27a1642aef03 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.436927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae620cc-1e4d-484b-a872-0e0bd351c0ba None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.464735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:52.478425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.479660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.479660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.479660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.481084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] RBAC: Adding query filter params (role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=4916304f10a44d759295c5da3237e502, user.id=0a20f1b54d6d470882694c7186cd6fd3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:52.481084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.482494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:52.489099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 98/190] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:52 2022] GET /identity/v3/role_assignments?user.id=0a20f1b54d6d470882694c7186cd6fd3&scope.project.id=4916304f10a44d759295c5da3237e502&role.id=1881f748eade41d6aa06dc2097202f09 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:52.489958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.527817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0230b94-2048-43a5-a7f7-27a1642aef03 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.567815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038d36d-c425-4d26-88ac-c153a341316a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.568819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038d36d-c425-4d26-88ac-c153a341316a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.569322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038d36d-c425-4d26-88ac-c153a341316a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.569766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038d36d-c425-4d26-88ac-c153a341316a None admin] PATH_INFO: `/v3/users/b77355cddbbd4853ad8d59fadfa3b34b` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.585663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038d36d-c425-4d26-88ac-c153a341316a None admin] RBAC: Authorizing `identity:get_user(user_id=b77355cddbbd4853ad8d59fadfa3b34b)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.587563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038d36d-c425-4d26-88ac-c153a341316a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:52.596093 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 93/191] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:52 2022] GET /identity/v3/users/b77355cddbbd4853ad8d59fadfa3b34b => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:52.599044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038d36d-c425-4d26-88ac-c153a341316a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.645537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76872d-69b7-43d6-8f14-da29fda943f5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.662815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:52.697078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.697634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.697905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.698216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.698894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:52.699201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.700606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:52.712428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 99/192] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:32:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:52.713690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.756253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038d36d-c425-4d26-88ac-c153a341316a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.771226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:52.795791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51c612-93aa-48d1-889b-c6d15f976820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.796415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c612-93aa-48d1-889b-c6d15f976820 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.796700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c612-93aa-48d1-889b-c6d15f976820 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.796976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c612-93aa-48d1-889b-c6d15f976820 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.797640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51c612-93aa-48d1-889b-c6d15f976820 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.799137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51c612-93aa-48d1-889b-c6d15f976820 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:52.824405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b891c-43be-40b0-a115-50c356a97d9b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:52.858590 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7d1c9-7b3b-416b-9fd3-73658bb54346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:52.859198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d1c9-7b3b-416b-9fd3-73658bb54346 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:52.859480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d1c9-7b3b-416b-9fd3-73658bb54346 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:52.859754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d1c9-7b3b-416b-9fd3-73658bb54346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:52.902074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:52.904637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 94/193] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:32:52 2022] POST /identity/v3/users => generated 297 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:52.905504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51c612-93aa-48d1-889b-c6d15f976820 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:52.929629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7d1c9-7b3b-416b-9fd3-73658bb54346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:52.931201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7d1c9-7b3b-416b-9fd3-73658bb54346 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:53.047038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51c612-93aa-48d1-889b-c6d15f976820 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:53.050617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 100/194] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:53.052200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e7e32-b245-4d63-a6e4-462cea2ad9a9 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:53.052666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e7e32-b245-4d63-a6e4-462cea2ad9a9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:53.053082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e7e32-b245-4d63-a6e4-462cea2ad9a9 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:53.054600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 101/195] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:53.059875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e7e32-b245-4d63-a6e4-462cea2ad9a9 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:53.090820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d901a-faf9-4dde-8abb-287b89945a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:53.092305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d901a-faf9-4dde-8abb-287b89945a44 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:53.092621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d901a-faf9-4dde-8abb-287b89945a44 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:53.093041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d901a-faf9-4dde-8abb-287b89945a44 None admin] PATH_INFO: `/v3/users/b77355cddbbd4853ad8d59fadfa3b34b` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:53.095400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d901a-faf9-4dde-8abb-287b89945a44 None admin] RBAC: Authorizing `identity:get_user(user_id=b77355cddbbd4853ad8d59fadfa3b34b)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:53.097017 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d901a-faf9-4dde-8abb-287b89945a44 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:53.098642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 95/196] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:52 2022] GET /identity/v3/users/b77355cddbbd4853ad8d59fadfa3b34b => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:53.099767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50810c7c-3ed0-457e-81f9-45ed12c5c639 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:53.100049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50810c7c-3ed0-457e-81f9-45ed12c5c639 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:53.100321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50810c7c-3ed0-457e-81f9-45ed12c5c639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:53.120060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-50810c7c-3ed0-457e-81f9-45ed12c5c639 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:53.123018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-50810c7c-3ed0-457e-81f9-45ed12c5c639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:53.152377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:53.161516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:53.179323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e7e32-b245-4d63-a6e4-462cea2ad9a9 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:53.212968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 96/197] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:53.215465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50810c7c-3ed0-457e-81f9-45ed12c5c639 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:53.226516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506433a-0c07-4840-aa44-cdb293b8089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:53.227657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506433a-0c07-4840-aa44-cdb293b8089f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:53.231289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506433a-0c07-4840-aa44-cdb293b8089f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:53.231289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506433a-0c07-4840-aa44-cdb293b8089f None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:53.231289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506433a-0c07-4840-aa44-cdb293b8089f None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:53.231289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506433a-0c07-4840-aa44-cdb293b8089f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:53.232839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 102/198] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:53 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:53.239067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506433a-0c07-4840-aa44-cdb293b8089f None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:53.358912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50810c7c-3ed0-457e-81f9-45ed12c5c639 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:53.403329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506433a-0c07-4840-aa44-cdb293b8089f None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:53.403866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52775c56-d5a3-4ab5-9e02-8fdd314b0f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:53.404109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775c56-d5a3-4ab5-9e02-8fdd314b0f32 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:53.404297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775c56-d5a3-4ab5-9e02-8fdd314b0f32 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:53.404572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775c56-d5a3-4ab5-9e02-8fdd314b0f32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:53.452788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:53.453402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:53.453677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:53.453948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:53.454943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=de79363b23ae49c99eab4dc709331b01, user.id=b77355cddbbd4853ad8d59fadfa3b34b) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:53.455424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:53.457255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:53.464766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 103/199] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:53 2022] GET /identity/v3/role_assignments?user.id=b77355cddbbd4853ad8d59fadfa3b34b&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=97f11873550e41dba0cd095afb1dec00 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:53.470195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:53.549456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52775c56-d5a3-4ab5-9e02-8fdd314b0f32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:53.551075 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52775c56-d5a3-4ab5-9e02-8fdd314b0f32 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:53.564998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:53.589371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:53.759527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d889c75-286b-4a38-9eef-91d51b0382f4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:53.801718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1167ac1-cdd1-4bfd-8a71-aee3fa7f36be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:53.802772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1167ac1-cdd1-4bfd-8a71-aee3fa7f36be None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:53.803272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1167ac1-cdd1-4bfd-8a71-aee3fa7f36be None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:53.803792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1167ac1-cdd1-4bfd-8a71-aee3fa7f36be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:53.818584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1167ac1-cdd1-4bfd-8a71-aee3fa7f36be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:53.824043 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 97/200] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-b1167ac1-cdd1-4bfd-8a71-aee3fa7f36be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:53.829787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:53.830839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:32:53.831635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:53.832242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52775c56-d5a3-4ab5-9e02-8fdd314b0f32 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:53.834424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 104/201] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:53.840574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1167ac1-cdd1-4bfd-8a71-aee3fa7f36be None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:53.901184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:53.978785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52775c56-d5a3-4ab5-9e02-8fdd314b0f32 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:54.027971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:54.041923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1795-4967-4e60-a1b0-bae5c16e7f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:54.044137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1795-4967-4e60-a1b0-bae5c16e7f6b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:54.044137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1795-4967-4e60-a1b0-bae5c16e7f6b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:54.044137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1795-4967-4e60-a1b0-bae5c16e7f6b None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:54.050182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df1795-4967-4e60-a1b0-bae5c16e7f6b None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:54.050182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df1795-4967-4e60-a1b0-bae5c16e7f6b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:54.050182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 98/202] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:53 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:54.057661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1795-4967-4e60-a1b0-bae5c16e7f6b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:54.106646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1167ac1-cdd1-4bfd-8a71-aee3fa7f36be None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:54.177117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:54.178078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:54.178614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:54.179082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:54.179979 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:54.180480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:54.182316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:54.192539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 105/203] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:54.198572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:54.220492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1795-4967-4e60-a1b0-bae5c16e7f6b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:54.283937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:54.291919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:54.292545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:54.292868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:54.293190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:54.294009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] RBAC: Adding query filter params (role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=de79363b23ae49c99eab4dc709331b01, user.id=b77355cddbbd4853ad8d59fadfa3b34b) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:54.294354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:54.295865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:54.313543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 99/204] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:54 2022] GET /identity/v3/role_assignments?user.id=b77355cddbbd4853ad8d59fadfa3b34b&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:32:54.397337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83591344-7690-4305-9de3-7bfee0b9fe61 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:54.443701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:54.453948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17317123-c07b-4cd9-aecf-d508b88013dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:54.454697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17317123-c07b-4cd9-aecf-d508b88013dd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:54.454986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17317123-c07b-4cd9-aecf-d508b88013dd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:54.455263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17317123-c07b-4cd9-aecf-d508b88013dd None admin] PATH_INFO: `/v3/users/a6087d73196047799351264a5820a57f` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:54.461069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17317123-c07b-4cd9-aecf-d508b88013dd None admin] RBAC: Authorizing `identity:get_user(user_id=a6087d73196047799351264a5820a57f)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:54.461069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17317123-c07b-4cd9-aecf-d508b88013dd None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:54.462642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 106/205] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:54 2022] GET /identity/v3/users/a6087d73196047799351264a5820a57f => generated 289 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:54.468562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:54.623533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459073a8-f120-4117-bcf2-5e7e044c7547 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:54.674328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128d7cb-4f7d-4d07-817a-e29ce5f7cee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:54.675081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128d7cb-4f7d-4d07-817a-e29ce5f7cee4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:54.675361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128d7cb-4f7d-4d07-817a-e29ce5f7cee4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:54.675633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128d7cb-4f7d-4d07-817a-e29ce5f7cee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:54.744077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:54.753611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128d7cb-4f7d-4d07-817a-e29ce5f7cee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:54.755166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128d7cb-4f7d-4d07-817a-e29ce5f7cee4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:54.906516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 100/206] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:54.907565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17317123-c07b-4cd9-aecf-d508b88013dd None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:55.095108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17317123-c07b-4cd9-aecf-d508b88013dd None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:55.162654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4301c-a5e0-41ae-8c86-5427031d4fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:55.163669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4301c-a5e0-41ae-8c86-5427031d4fc1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:55.164168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4301c-a5e0-41ae-8c86-5427031d4fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:55.164588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4301c-a5e0-41ae-8c86-5427031d4fc1 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:55.166488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4301c-a5e0-41ae-8c86-5427031d4fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:55.168800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4301c-a5e0-41ae-8c86-5427031d4fc1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:55.176791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 107/207] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:54 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:55.179271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128d7cb-4f7d-4d07-817a-e29ce5f7cee4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:55.350878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:55.359715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128d7cb-4f7d-4d07-817a-e29ce5f7cee4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:55.418391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33d46a-e344-436f-b069-e032cef37f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:55.419233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d46a-e344-436f-b069-e032cef37f4f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:55.419601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d46a-e344-436f-b069-e032cef37f4f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:55.419988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d46a-e344-436f-b069-e032cef37f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:55.421284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33d46a-e344-436f-b069-e032cef37f4f None admin] RBAC: Adding query filter params (role.id=44854c86be404c76a5703d725444f549, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2, user.id=a6087d73196047799351264a5820a57f) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:55.421660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33d46a-e344-436f-b069-e032cef37f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:55.423470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33d46a-e344-436f-b069-e032cef37f4f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:55.434395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 101/208] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:55 2022] GET /identity/v3/role_assignments?user.id=a6087d73196047799351264a5820a57f&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=44854c86be404c76a5703d725444f549 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:56.027429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf635-9b99-4259-8703-96d8197675ea None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.027937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf635-9b99-4259-8703-96d8197675ea None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.028014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf635-9b99-4259-8703-96d8197675ea None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.030011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 108/209] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:56.037354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60555f6f-622a-48e7-94d0-02a832765e3e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.038340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60555f6f-622a-48e7-94d0-02a832765e3e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.038857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60555f6f-622a-48e7-94d0-02a832765e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.067408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-60555f6f-622a-48e7-94d0-02a832765e3e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:56.069065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-60555f6f-622a-48e7-94d0-02a832765e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:56.158950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 102/210] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:56.164751 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36933-8a13-4cf8-bd5e-c95bc7f6e1f9 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.165625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36933-8a13-4cf8-bd5e-c95bc7f6e1f9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.166443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36933-8a13-4cf8-bd5e-c95bc7f6e1f9 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.169191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 109/211] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:56.176402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8554b-0860-4379-89be-f55b7d5f7d98 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.177009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8554b-0860-4379-89be-f55b7d5f7d98 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.177481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8554b-0860-4379-89be-f55b7d5f7d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.192699 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f250d-4ace-4bc7-be4c-0b0928649d1e None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.193370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f250d-4ace-4bc7-be4c-0b0928649d1e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.193931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f250d-4ace-4bc7-be4c-0b0928649d1e None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.195616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 110/212] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:32:56.203218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-a1b8554b-0860-4379-89be-f55b7d5f7d98 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:56.203423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a1b8554b-0860-4379-89be-f55b7d5f7d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:56.203795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874d419-2791-4a6d-9b46-25d4a340b19a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.206337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874d419-2791-4a6d-9b46-25d4a340b19a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.206957 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874d419-2791-4a6d-9b46-25d4a340b19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.237553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a874d419-2791-4a6d-9b46-25d4a340b19a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:56.240773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a874d419-2791-4a6d-9b46-25d4a340b19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:56.292479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 103/213] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:56.322186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:56.332573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 111/214] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:56.488754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8554b-0860-4379-89be-f55b7d5f7d98 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:56.511942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:56.591311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874d419-2791-4a6d-9b46-25d4a340b19a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:56.603203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8554b-0860-4379-89be-f55b7d5f7d98 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:56.650331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650a85d-6986-4724-9d42-b2b6cb63ff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:56.651255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650a85d-6986-4724-9d42-b2b6cb63ff94 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.651799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650a85d-6986-4724-9d42-b2b6cb63ff94 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.651904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650a85d-6986-4724-9d42-b2b6cb63ff94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.652983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650a85d-6986-4724-9d42-b2b6cb63ff94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:56.654500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650a85d-6986-4724-9d42-b2b6cb63ff94 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:56.656559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 104/215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:32:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:56.657775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650a85d-6986-4724-9d42-b2b6cb63ff94 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:56.727770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874d419-2791-4a6d-9b46-25d4a340b19a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:56.764101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:56.766462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3ba02-c658-4bfa-bf79-66f20589b09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:56.767108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3ba02-c658-4bfa-bf79-66f20589b09a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.767398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3ba02-c658-4bfa-bf79-66f20589b09a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.767650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3ba02-c658-4bfa-bf79-66f20589b09a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.776721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650a85d-6986-4724-9d42-b2b6cb63ff94 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:56.778373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3ba02-c658-4bfa-bf79-66f20589b09a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-68a3ba02-c658-4bfa-bf79-66f20589b09a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:56.784269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:56.785213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:56.786015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 112/216] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:32:56.787288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3ba02-c658-4bfa-bf79-66f20589b09a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:56.819738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7d63e-df7b-4a52-aa84-e20d51e1590e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:56.820534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7d63e-df7b-4a52-aa84-e20d51e1590e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.820814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7d63e-df7b-4a52-aa84-e20d51e1590e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.821135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7d63e-df7b-4a52-aa84-e20d51e1590e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.833011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7d63e-df7b-4a52-aa84-e20d51e1590e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-5af7d63e-df7b-4a52-aa84-e20d51e1590e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:56.838940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:56.839862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 105/217] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:32:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:56.845906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7d63e-df7b-4a52-aa84-e20d51e1590e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:56.864595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:56.918656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3ba02-c658-4bfa-bf79-66f20589b09a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:56.956648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f326ee6-a673-448c-95bc-b706563e39a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:56.957501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f326ee6-a673-448c-95bc-b706563e39a3 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:56.957974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f326ee6-a673-448c-95bc-b706563e39a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:56.958291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f326ee6-a673-448c-95bc-b706563e39a3 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:56.959244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f326ee6-a673-448c-95bc-b706563e39a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:56.960788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f326ee6-a673-448c-95bc-b706563e39a3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:56.966809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7d63e-df7b-4a52-aa84-e20d51e1590e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:57.004757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:57.005442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:57.005635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:57.005906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:57.006685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:57.006985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:57.008367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:57.017200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 106/218] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:32:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:57.018040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:57.033036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:57.089641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 113/219] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:32:56 2022] POST /identity/v3/users => generated 297 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:32:57.095631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f326ee6-a673-448c-95bc-b706563e39a3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:57.144523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad828-87ff-49bc-80e2-bf9ea33471b4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:57.191579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:57.192817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:57.193978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:57.194540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:57.195799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:57.196331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:57.199288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:57.210758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 107/220] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:32:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:57.218772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:57.239913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f326ee6-a673-448c-95bc-b706563e39a3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:57.254731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:57.285125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ba4cc-6217-48f9-9d8d-185e03600d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:57.285775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba4cc-6217-48f9-9d8d-185e03600d2b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:57.286056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba4cc-6217-48f9-9d8d-185e03600d2b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:57.286371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba4cc-6217-48f9-9d8d-185e03600d2b None admin] PATH_INFO: `/v3/users/d4758c8f613f4e48b00ea834419d963e` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:57.303034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ba4cc-6217-48f9-9d8d-185e03600d2b None admin] RBAC: Authorizing `identity:get_user(user_id=d4758c8f613f4e48b00ea834419d963e)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:57.304418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ba4cc-6217-48f9-9d8d-185e03600d2b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:57.310169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 114/221] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:57 2022] GET /identity/v3/users/d4758c8f613f4e48b00ea834419d963e => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:57.310169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6922-9721-4043-ad9b-bb72bd684f58 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:57.310169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6922-9721-4043-ad9b-bb72bd684f58 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:57.310169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6922-9721-4043-ad9b-bb72bd684f58 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:57.312181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 115/222] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:57.314430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c6922-9721-4043-ad9b-bb72bd684f58 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:57.327114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:57.365099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc71e03-1881-47df-8a54-1584dfff586f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:57.410802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1e221-421d-4c62-9da9-a818798ad520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:57.411736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e221-421d-4c62-9da9-a818798ad520 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:57.412226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e221-421d-4c62-9da9-a818798ad520 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:57.412514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e221-421d-4c62-9da9-a818798ad520 None admin] PATH_INFO: `/v3/users/b77355cddbbd4853ad8d59fadfa3b34b` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:57.413803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1e221-421d-4c62-9da9-a818798ad520 None admin] RBAC: Authorizing `identity:get_user(user_id=b77355cddbbd4853ad8d59fadfa3b34b)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:57.415920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1e221-421d-4c62-9da9-a818798ad520 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:57.417849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 108/223] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:57 2022] GET /identity/v3/users/b77355cddbbd4853ad8d59fadfa3b34b => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:57.418980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153276f-982d-40af-b415-b5b1c9be1cb4 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:57.419366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153276f-982d-40af-b415-b5b1c9be1cb4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:57.419644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153276f-982d-40af-b415-b5b1c9be1cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:57.444225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b153276f-982d-40af-b415-b5b1c9be1cb4 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:57.445839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b153276f-982d-40af-b415-b5b1c9be1cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:57.466278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c6922-9721-4043-ad9b-bb72bd684f58 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:57.513063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63c3cf-2d4e-4af2-b77a-954f21730182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:57.513655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63c3cf-2d4e-4af2-b77a-954f21730182 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:57.513937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63c3cf-2d4e-4af2-b77a-954f21730182 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:57.514239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63c3cf-2d4e-4af2-b77a-954f21730182 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:57.540969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 109/224] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:57.541846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153276f-982d-40af-b415-b5b1c9be1cb4 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:57.557266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:57.629824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63c3cf-2d4e-4af2-b77a-954f21730182 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:57.633893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63c3cf-2d4e-4af2-b77a-954f21730182 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:57.714883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:57.789204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 116/225] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:32:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:32:57.795127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63c3cf-2d4e-4af2-b77a-954f21730182 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:57.808009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153276f-982d-40af-b415-b5b1c9be1cb4 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:57.882522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ecd4d-9456-46bf-a40e-961c81c7f2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:57.883405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ecd4d-9456-46bf-a40e-961c81c7f2a1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:57.884183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ecd4d-9456-46bf-a40e-961c81c7f2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:57.884601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ecd4d-9456-46bf-a40e-961c81c7f2a1 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:57.886183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ecd4d-9456-46bf-a40e-961c81c7f2a1 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:57.887913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ecd4d-9456-46bf-a40e-961c81c7f2a1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:57.894690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 110/226] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:57 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:57.895934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ecd4d-9456-46bf-a40e-961c81c7f2a1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:57.925118 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:57.961454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63c3cf-2d4e-4af2-b77a-954f21730182 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:58.000985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7117c02-dd5e-47cf-bbe2-36c371e50161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:58.001643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117c02-dd5e-47cf-bbe2-36c371e50161 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:58.002026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117c02-dd5e-47cf-bbe2-36c371e50161 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:58.002378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117c02-dd5e-47cf-bbe2-36c371e50161 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:58.003549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7117c02-dd5e-47cf-bbe2-36c371e50161 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:58.005232 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7117c02-dd5e-47cf-bbe2-36c371e50161 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:58.007036 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 117/227] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:57 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:58.007900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7117c02-dd5e-47cf-bbe2-36c371e50161 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:58.015834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ecd4d-9456-46bf-a40e-961c81c7f2a1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:58.053027 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37ffcb-1707-4851-92ac-da003aa4405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:58.053819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37ffcb-1707-4851-92ac-da003aa4405c None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:58.054096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37ffcb-1707-4851-92ac-da003aa4405c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:58.054397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37ffcb-1707-4851-92ac-da003aa4405c None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01/users/b77355cddbbd4853ad8d59fadfa3b34b/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:58.056425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37ffcb-1707-4851-92ac-da003aa4405c None admin] RBAC: Authorizing `identity:create_grant(project_id=de79363b23ae49c99eab4dc709331b01, user_id=b77355cddbbd4853ad8d59fadfa3b34b, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:58.057990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37ffcb-1707-4851-92ac-da003aa4405c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:58.116491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7117c02-dd5e-47cf-bbe2-36c371e50161 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:58.128043 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:58.135099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 111/228] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:32:57 2022] PUT /identity/v3/projects/de79363b23ae49c99eab4dc709331b01/users/b77355cddbbd4853ad8d59fadfa3b34b/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:32:58.136403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37ffcb-1707-4851-92ac-da003aa4405c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:58.171633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dfbd9-0d56-4ed9-bc86-b9a7d6d81cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:58.172376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dfbd9-0d56-4ed9-bc86-b9a7d6d81cc9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:58.172669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dfbd9-0d56-4ed9-bc86-b9a7d6d81cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:58.173077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dfbd9-0d56-4ed9-bc86-b9a7d6d81cc9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:58.179848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:58.203683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dfbd9-0d56-4ed9-bc86-b9a7d6d81cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-b02dfbd9-0d56-4ed9-bc86-b9a7d6d81cc9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:32:58.208576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:32:58.209480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:32:58.210309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 118/229] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:32:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:32:58.219408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dfbd9-0d56-4ed9-bc86-b9a7d6d81cc9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:58.273229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37ffcb-1707-4851-92ac-da003aa4405c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:58.314795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:58.315928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:58.316408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:58.316903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:58.317845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] RBAC: Adding query filter params (role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=de79363b23ae49c99eab4dc709331b01, user.id=d4758c8f613f4e48b00ea834419d963e) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:58.318379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:58.320091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:58.328772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 112/230] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:32:58 2022] GET /identity/v3/role_assignments?user.id=d4758c8f613f4e48b00ea834419d963e&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=1881f748eade41d6aa06dc2097202f09 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:58.448299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dfbd9-0d56-4ed9-bc86-b9a7d6d81cc9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:58.491938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:58.492581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:58.492864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:58.493139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:58.493831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:32:58.494162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:58.495584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:58.507376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 119/231] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:32:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:58.513046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:58.586955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:58.627333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bd60-d5cf-4499-9fad-b6e7592c7a31 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:58.662984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb563642-d51f-43fa-9177-ba8f2345eee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:58.663626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb563642-d51f-43fa-9177-ba8f2345eee6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:58.663905 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb563642-d51f-43fa-9177-ba8f2345eee6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:58.664174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb563642-d51f-43fa-9177-ba8f2345eee6 None admin] PATH_INFO: `/v3/users/b77355cddbbd4853ad8d59fadfa3b34b` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:58.665315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb563642-d51f-43fa-9177-ba8f2345eee6 None admin] RBAC: Authorizing `identity:get_user(user_id=b77355cddbbd4853ad8d59fadfa3b34b)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:58.666795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb563642-d51f-43fa-9177-ba8f2345eee6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:58.668847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 113/232] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:32:58 2022] GET /identity/v3/users/b77355cddbbd4853ad8d59fadfa3b34b => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:58.674238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:58.768201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:58.811913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a79da-53ff-4062-982c-7a55e09e0f49 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:58.863560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91712d89-50e7-4aae-9362-33a9ddf64f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:58.864121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91712d89-50e7-4aae-9362-33a9ddf64f38 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:58.864401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91712d89-50e7-4aae-9362-33a9ddf64f38 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:58.864678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91712d89-50e7-4aae-9362-33a9ddf64f38 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:58.865794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91712d89-50e7-4aae-9362-33a9ddf64f38 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:58.867284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91712d89-50e7-4aae-9362-33a9ddf64f38 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:58.869215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 120/233] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:32:58 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:32:58.875382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb563642-d51f-43fa-9177-ba8f2345eee6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:32:59.004424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb563642-d51f-43fa-9177-ba8f2345eee6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:32:59.035758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:32:59.042081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ff14d-d0b1-4d55-87fa-2e0bb0c70fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:32:59.042974 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff14d-d0b1-4d55-87fa-2e0bb0c70fbe None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:59.043415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff14d-d0b1-4d55-87fa-2e0bb0c70fbe None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:59.044001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff14d-d0b1-4d55-87fa-2e0bb0c70fbe None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01/users/b77355cddbbd4853ad8d59fadfa3b34b/roles/4ed4b4b515ca4823abbe45566fe27161` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:59.046913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ff14d-d0b1-4d55-87fa-2e0bb0c70fbe None admin] RBAC: Authorizing `identity:create_grant(project_id=de79363b23ae49c99eab4dc709331b01, user_id=b77355cddbbd4853ad8d59fadfa3b34b, role_id=4ed4b4b515ca4823abbe45566fe27161)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:32:59.048512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ff14d-d0b1-4d55-87fa-2e0bb0c70fbe None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:32:59.193048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 114/234] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:32:58 2022] PUT /identity/v3/projects/de79363b23ae49c99eab4dc709331b01/users/b77355cddbbd4853ad8d59fadfa3b34b/roles/4ed4b4b515ca4823abbe45566fe27161 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:32:59.761791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939338d4-67b0-4006-adf1-95418f3eea8d None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:59.762310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939338d4-67b0-4006-adf1-95418f3eea8d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:59.762385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939338d4-67b0-4006-adf1-95418f3eea8d None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:59.763699 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 121/235] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:59.770124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e64a08-d3c2-415c-92f4-5b275feec81f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:59.770590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e64a08-d3c2-415c-92f4-5b275feec81f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:59.770768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e64a08-d3c2-415c-92f4-5b275feec81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:59.789183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a60f4-7c9d-4502-a7f1-c2672ecc0372 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:59.789507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a60f4-7c9d-4502-a7f1-c2672ecc0372 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:59.789748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a60f4-7c9d-4502-a7f1-c2672ecc0372 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:59.791156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 122/236] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:32:59.794948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-63e64a08-d3c2-415c-92f4-5b275feec81f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:59.796232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-63e64a08-d3c2-415c-92f4-5b275feec81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:59.797115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d577-f033-4dfb-b352-1c119dfd256f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:32:59.797553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d577-f033-4dfb-b352-1c119dfd256f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:32:59.797993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d577-f033-4dfb-b352-1c119dfd256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:32:59.831202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f9e3d577-f033-4dfb-b352-1c119dfd256f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:32:59.832663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f9e3d577-f033-4dfb-b352-1c119dfd256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:32:59.876144 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 115/237] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:32:59.926215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 123/238] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:00.170589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e64a08-d3c2-415c-92f4-5b275feec81f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:00.201917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3d577-f033-4dfb-b352-1c119dfd256f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:00.261615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:00.437361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e64a08-d3c2-415c-92f4-5b275feec81f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:00.442881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3d577-f033-4dfb-b352-1c119dfd256f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:00.482055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:00.510264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c10757-7837-49c0-8c4e-1618ec38a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:00.510905 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c10757-7837-49c0-8c4e-1618ec38a115 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:00.511186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c10757-7837-49c0-8c4e-1618ec38a115 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:00.511452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c10757-7837-49c0-8c4e-1618ec38a115 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:00.525697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1f9b0-6639-43ce-8755-10ad0bb17d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:00.526385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f9b0-6639-43ce-8755-10ad0bb17d26 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:00.526665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f9b0-6639-43ce-8755-10ad0bb17d26 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:00.530055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f9b0-6639-43ce-8755-10ad0bb17d26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:00.533567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1f9b0-6639-43ce-8755-10ad0bb17d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:00.536168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1f9b0-6639-43ce-8755-10ad0bb17d26 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:00.539905 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c10757-7837-49c0-8c4e-1618ec38a115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-17c10757-7837-49c0-8c4e-1618ec38a115 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:00.543556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:00.544514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:00.545308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 116/239] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:00.549692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 124/240] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:00.551895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c10757-7837-49c0-8c4e-1618ec38a115 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:00.554992 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1f9b0-6639-43ce-8755-10ad0bb17d26 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:00.695147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1f9b0-6639-43ce-8755-10ad0bb17d26 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:00.725707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:00.745411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25653e-369c-4c3e-8438-42fa77b2bc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:00.745411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25653e-369c-4c3e-8438-42fa77b2bc97 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:00.745411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25653e-369c-4c3e-8438-42fa77b2bc97 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:00.745411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25653e-369c-4c3e-8438-42fa77b2bc97 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:00.745839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25653e-369c-4c3e-8438-42fa77b2bc97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:00.745839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25653e-369c-4c3e-8438-42fa77b2bc97 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:00.809025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c10757-7837-49c0-8c4e-1618ec38a115 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:00.838828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 125/241] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:33:00 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:33:00.868399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b530a-99b7-405a-903b-51a57ad41fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:00.869036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b530a-99b7-405a-903b-51a57ad41fbe None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:00.869316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b530a-99b7-405a-903b-51a57ad41fbe None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:00.869588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b530a-99b7-405a-903b-51a57ad41fbe None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:00.870451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b530a-99b7-405a-903b-51a57ad41fbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:00.870762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b530a-99b7-405a-903b-51a57ad41fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:00.872292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b530a-99b7-405a-903b-51a57ad41fbe None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:00.883881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 117/242] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:00.890238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25653e-369c-4c3e-8438-42fa77b2bc97 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:01.033808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412ac7-328b-464f-b16b-a53eb25f29ae None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:01.034112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412ac7-328b-464f-b16b-a53eb25f29ae None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:01.034310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412ac7-328b-464f-b16b-a53eb25f29ae None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:01.036004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 118/243] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:01.042088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560af08-c0ff-459a-88dc-1f1cefe17df9 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:01.042739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560af08-c0ff-459a-88dc-1f1cefe17df9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:01.043208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560af08-c0ff-459a-88dc-1f1cefe17df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:01.086071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f560af08-c0ff-459a-88dc-1f1cefe17df9 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:01.094282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f560af08-c0ff-459a-88dc-1f1cefe17df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:01.117137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25653e-369c-4c3e-8438-42fa77b2bc97 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:01.162535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:01.163548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0123-c84c-4c01-89c1-1453f23fefde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:01.164433 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0123-c84c-4c01-89c1-1453f23fefde None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:01.164905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0123-c84c-4c01-89c1-1453f23fefde None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:01.165392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0123-c84c-4c01-89c1-1453f23fefde None admin] PATH_INFO: `/v3/users/b89c4a3ab4f24760a6feaf9d1a367732` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:01.184117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac0123-c84c-4c01-89c1-1453f23fefde None admin] RBAC: Authorizing `identity:get_user(user_id=b89c4a3ab4f24760a6feaf9d1a367732)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:01.186077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac0123-c84c-4c01-89c1-1453f23fefde None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:01.188717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 126/244] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:00 2022] GET /identity/v3/users/b89c4a3ab4f24760a6feaf9d1a367732 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:01.194726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0123-c84c-4c01-89c1-1453f23fefde None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:01.212819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 119/245] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:01.214685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b219dc-f70f-4bc9-8c15-9505541e66cf None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:01.215047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b219dc-f70f-4bc9-8c15-9505541e66cf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:01.215394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b219dc-f70f-4bc9-8c15-9505541e66cf None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:01.221594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 120/246] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:01.223580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93a697-c1ac-4786-9589-eb8a2292c1d9 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:01.223939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93a697-c1ac-4786-9589-eb8a2292c1d9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:01.224313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93a697-c1ac-4786-9589-eb8a2292c1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:01.244872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-be93a697-c1ac-4786-9589-eb8a2292c1d9 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:01.246256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-be93a697-c1ac-4786-9589-eb8a2292c1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:01.264795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:01.332795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 121/247] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:01.351226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0123-c84c-4c01-89c1-1453f23fefde None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:01.394028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dac9c-5fef-48ac-a425-bedaadfb1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:01.395049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dac9c-5fef-48ac-a425-bedaadfb1d2c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:01.395545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dac9c-5fef-48ac-a425-bedaadfb1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:01.396052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dac9c-5fef-48ac-a425-bedaadfb1d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:01.488611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01dac9c-5fef-48ac-a425-bedaadfb1d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:01.490523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01dac9c-5fef-48ac-a425-bedaadfb1d2c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:01.545567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:01.632642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93a697-c1ac-4786-9589-eb8a2292c1d9 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:01.644277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 127/248] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:01.651113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dac9c-5fef-48ac-a425-bedaadfb1d2c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:01.799707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93a697-c1ac-4786-9589-eb8a2292c1d9 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:01.802668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:01.820038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dac9c-5fef-48ac-a425-bedaadfb1d2c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:01.869972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea30eb-1596-4e94-af31-03e6ad2623b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:01.871504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea30eb-1596-4e94-af31-03e6ad2623b6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:01.871980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea30eb-1596-4e94-af31-03e6ad2623b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:01.872446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea30eb-1596-4e94-af31-03e6ad2623b6 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:01.874627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea30eb-1596-4e94-af31-03e6ad2623b6 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:01.877815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea30eb-1596-4e94-af31-03e6ad2623b6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:01.883619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 128/249] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:01 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:01.885866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea30eb-1596-4e94-af31-03e6ad2623b6 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:01.902274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:01.905154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3979bc3-d6bb-416f-aacb-a04db9d41eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:01.906354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3979bc3-d6bb-416f-aacb-a04db9d41eea None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:01.906918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3979bc3-d6bb-416f-aacb-a04db9d41eea None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:01.907329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3979bc3-d6bb-416f-aacb-a04db9d41eea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:01.947772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3979bc3-d6bb-416f-aacb-a04db9d41eea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b3979bc3-d6bb-416f-aacb-a04db9d41eea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:01.965079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:01.966397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:01.967486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:01.968134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 122/250] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:33:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:01.969348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3979bc3-d6bb-416f-aacb-a04db9d41eea None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:02.015586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea30eb-1596-4e94-af31-03e6ad2623b6 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:02.065626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bef492-595e-4992-a8bc-3a7c8725ca2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:02.066805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bef492-595e-4992-a8bc-3a7c8725ca2e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.067295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bef492-595e-4992-a8bc-3a7c8725ca2e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.068640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bef492-595e-4992-a8bc-3a7c8725ca2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:02.086916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bef492-595e-4992-a8bc-3a7c8725ca2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-d7bef492-595e-4992-a8bc-3a7c8725ca2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:02.094117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:02.095048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:02.095840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:02.098702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 129/251] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:02.099737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bef492-595e-4992-a8bc-3a7c8725ca2e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:02.181075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:02.202059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3979bc3-d6bb-416f-aacb-a04db9d41eea None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:02.232866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bef492-595e-4992-a8bc-3a7c8725ca2e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:02.250279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:02.251589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.252587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.253415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:02.254487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] RBAC: Adding query filter params (role.id=44854c86be404c76a5703d725444f549, scope.project.id=de79363b23ae49c99eab4dc709331b01, user.id=b89c4a3ab4f24760a6feaf9d1a367732) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:02.255257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:02.257430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:02.266934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 123/252] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:01 2022] GET /identity/v3/role_assignments?user.id=b89c4a3ab4f24760a6feaf9d1a367732&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=44854c86be404c76a5703d725444f549 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:02.268422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:02.285895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2d23b-9132-4a64-8e91-c7ac3e711a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:02.285895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2d23b-9132-4a64-8e91-c7ac3e711a41 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.285895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2d23b-9132-4a64-8e91-c7ac3e711a41 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.285895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2d23b-9132-4a64-8e91-c7ac3e711a41 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:02.286334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2d23b-9132-4a64-8e91-c7ac3e711a41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:02.286334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2d23b-9132-4a64-8e91-c7ac3e711a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:02.290400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2d23b-9132-4a64-8e91-c7ac3e711a41 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:02.304963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 130/253] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:02.306323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f51a3b-f6a5-4a89-93e1-1798216185c0 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.306613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f51a3b-f6a5-4a89-93e1-1798216185c0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.306889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f51a3b-f6a5-4a89-93e1-1798216185c0 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:02.308226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 131/254] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:02.311803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f51a3b-f6a5-4a89-93e1-1798216185c0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:02.336414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:02.517686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ca786-d258-4657-bf47-1fade9353a4b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:02.539038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f51a3b-f6a5-4a89-93e1-1798216185c0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:02.560836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39547c23-bd42-4e7b-b2ca-7ada96caa316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:02.562374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39547c23-bd42-4e7b-b2ca-7ada96caa316 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.562934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39547c23-bd42-4e7b-b2ca-7ada96caa316 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.563406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39547c23-bd42-4e7b-b2ca-7ada96caa316 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:02.564357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39547c23-bd42-4e7b-b2ca-7ada96caa316 None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:02.564876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39547c23-bd42-4e7b-b2ca-7ada96caa316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:02.566799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39547c23-bd42-4e7b-b2ca-7ada96caa316 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:02.576986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 124/255] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:02.579996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52de77-3371-447e-960b-0f13f4856100 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.580644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52de77-3371-447e-960b-0f13f4856100 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.581160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52de77-3371-447e-960b-0f13f4856100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:02.594976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:02.613757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd757a-a43b-4bd9-932b-671d06084504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:02.614715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd757a-a43b-4bd9-932b-671d06084504 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.615002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd757a-a43b-4bd9-932b-671d06084504 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.615280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd757a-a43b-4bd9-932b-671d06084504 None admin] PATH_INFO: `/v3/users/b77355cddbbd4853ad8d59fadfa3b34b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:02.617171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd757a-a43b-4bd9-932b-671d06084504 None admin] RBAC: Authorizing `identity:get_user(user_id=b77355cddbbd4853ad8d59fadfa3b34b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:02.618886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9c52de77-3371-447e-960b-0f13f4856100 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:02.619791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd757a-a43b-4bd9-932b-671d06084504 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:02.621446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 132/256] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:02 2022] GET /identity/v3/users/b77355cddbbd4853ad8d59fadfa3b34b => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:02.622573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9c52de77-3371-447e-960b-0f13f4856100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:02.625151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd757a-a43b-4bd9-932b-671d06084504 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:02.735870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 125/257] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:02.736763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52de77-3371-447e-960b-0f13f4856100 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:02.774667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:02.814772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd757a-a43b-4bd9-932b-671d06084504 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:02.862873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe045-b492-4da2-88c8-707068b068f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:02.863536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe045-b492-4da2-88c8-707068b068f3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.863820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe045-b492-4da2-88c8-707068b068f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.864096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe045-b492-4da2-88c8-707068b068f3 None admin] PATH_INFO: `/v3/users/d4758c8f613f4e48b00ea834419d963e` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:02.865980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffe045-b492-4da2-88c8-707068b068f3 None admin] RBAC: Authorizing `identity:get_user(user_id=d4758c8f613f4e48b00ea834419d963e)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:02.867476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffe045-b492-4da2-88c8-707068b068f3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:02.870322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 133/258] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:02 2022] GET /identity/v3/users/d4758c8f613f4e48b00ea834419d963e => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:02.875596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe045-b492-4da2-88c8-707068b068f3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:02.955130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52de77-3371-447e-960b-0f13f4856100 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:02.995593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d9778-76b6-4876-8e0d-1ae52c6ca90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:02.996216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d9778-76b6-4876-8e0d-1ae52c6ca90c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:02.996499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d9778-76b6-4876-8e0d-1ae52c6ca90c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:02.996775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d9778-76b6-4876-8e0d-1ae52c6ca90c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:03.003169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:03.098053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d9778-76b6-4876-8e0d-1ae52c6ca90c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:03.099911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d9778-76b6-4876-8e0d-1ae52c6ca90c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:03.107473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe045-b492-4da2-88c8-707068b068f3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:03.127493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:03.201876 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1987389-3eec-4055-a160-876ab97fa182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:03.203216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1987389-3eec-4055-a160-876ab97fa182 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:03.203499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1987389-3eec-4055-a160-876ab97fa182 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:03.203774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1987389-3eec-4055-a160-876ab97fa182 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:03.205013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1987389-3eec-4055-a160-876ab97fa182 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:03.206577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1987389-3eec-4055-a160-876ab97fa182 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:03.213484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 134/259] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:02 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:03.219096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1987389-3eec-4055-a160-876ab97fa182 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:03.243525 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 126/260] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:03.250124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d9778-76b6-4876-8e0d-1ae52c6ca90c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:03.355695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d9778-76b6-4876-8e0d-1ae52c6ca90c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:03.378563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:03.393951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37117f-3551-4f5f-a712-91e4ded34ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:03.394935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37117f-3551-4f5f-a712-91e4ded34ddb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:03.395385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37117f-3551-4f5f-a712-91e4ded34ddb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:03.395835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37117f-3551-4f5f-a712-91e4ded34ddb None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:03.397158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37117f-3551-4f5f-a712-91e4ded34ddb None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:03.399218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37117f-3551-4f5f-a712-91e4ded34ddb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:03.400308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1987389-3eec-4055-a160-876ab97fa182 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:03.401347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 127/261] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:03 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:03.407092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37117f-3551-4f5f-a712-91e4ded34ddb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:03.491123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:03.514499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37117f-3551-4f5f-a712-91e4ded34ddb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:03.514872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fab2b-2c97-4bca-99b4-317c49982457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:03.515562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fab2b-2c97-4bca-99b4-317c49982457 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:03.515877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fab2b-2c97-4bca-99b4-317c49982457 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:03.516179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fab2b-2c97-4bca-99b4-317c49982457 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01/users/d4758c8f613f4e48b00ea834419d963e/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:03.526943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fab2b-2c97-4bca-99b4-317c49982457 None admin] RBAC: Authorizing `identity:create_grant(project_id=de79363b23ae49c99eab4dc709331b01, user_id=d4758c8f613f4e48b00ea834419d963e, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:03.528411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fab2b-2c97-4bca-99b4-317c49982457 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:03.556067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83319-f9a9-43d2-96f2-94a891a35edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:03.556708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83319-f9a9-43d2-96f2-94a891a35edb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:03.556990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83319-f9a9-43d2-96f2-94a891a35edb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:03.557298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83319-f9a9-43d2-96f2-94a891a35edb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:03.568800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e83319-f9a9-43d2-96f2-94a891a35edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-c6e83319-f9a9-43d2-96f2-94a891a35edb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:03.574329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:03.575229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:03.576423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 128/262] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:33:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:03.577329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83319-f9a9-43d2-96f2-94a891a35edb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:03.615212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 135/263] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:33:03 2022] PUT /identity/v3/projects/de79363b23ae49c99eab4dc709331b01/users/d4758c8f613f4e48b00ea834419d963e/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:03.615972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fab2b-2c97-4bca-99b4-317c49982457 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:03.668522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:03.745792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83319-f9a9-43d2-96f2-94a891a35edb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:03.814948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:03.815579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:03.815857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:03.816131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:03.817131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=de79363b23ae49c99eab4dc709331b01, user.id=b77355cddbbd4853ad8d59fadfa3b34b) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:03.817448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:03.818921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:03.826042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 129/264] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:03 2022] GET /identity/v3/role_assignments?user.id=b77355cddbbd4853ad8d59fadfa3b34b&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=97f11873550e41dba0cd095afb1dec00 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:03.907536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fab2b-2c97-4bca-99b4-317c49982457 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:03.957157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:03.957817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:03.958097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:03.958403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:03.959100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:03.959407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:03.960854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:03.976221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 136/265] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:03.982171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:04.158262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e6715-a1d6-4c98-b722-0ea1f926ebcc None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:04.206549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:04.215826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8072ea8-9801-46b2-9f6a-5d7574f03571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:04.216942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072ea8-9801-46b2-9f6a-5d7574f03571 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.217707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072ea8-9801-46b2-9f6a-5d7574f03571 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.218780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072ea8-9801-46b2-9f6a-5d7574f03571 None admin] PATH_INFO: `/v3/users/b77355cddbbd4853ad8d59fadfa3b34b` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:04.220953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8072ea8-9801-46b2-9f6a-5d7574f03571 None admin] RBAC: Authorizing `identity:get_user(user_id=b77355cddbbd4853ad8d59fadfa3b34b)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:04.223625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8072ea8-9801-46b2-9f6a-5d7574f03571 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:04.225865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 130/266] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:03 2022] GET /identity/v3/users/b77355cddbbd4853ad8d59fadfa3b34b => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:04.235347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:04.328476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:04.378766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0922a4-d4b4-4e8a-8b3e-9b1a59f581fe None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:04.429158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73dd4b-a8e9-44f1-922d-c8d13d4e3a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:04.429666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73dd4b-a8e9-44f1-922d-c8d13d4e3a4c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.429946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73dd4b-a8e9-44f1-922d-c8d13d4e3a4c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.430250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73dd4b-a8e9-44f1-922d-c8d13d4e3a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:04.518468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73dd4b-a8e9-44f1-922d-c8d13d4e3a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:04.520002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73dd4b-a8e9-44f1-922d-c8d13d4e3a4c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:04.599052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:04.630579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c9b7f-959c-454d-b6e3-aa3ce9d96180 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.630897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c9b7f-959c-454d-b6e3-aa3ce9d96180 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.631136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c9b7f-959c-454d-b6e3-aa3ce9d96180 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:04.632537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 131/267] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:33:04.642672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8f6d5-abf0-452e-873b-d53df86eaa5d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.642994 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8f6d5-abf0-452e-873b-d53df86eaa5d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.643223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8f6d5-abf0-452e-873b-d53df86eaa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:04.664716 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0ce8f6d5-abf0-452e-873b-d53df86eaa5d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:04.666176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0ce8f6d5-abf0-452e-873b-d53df86eaa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:04.673890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 137/268] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:04.679538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73dd4b-a8e9-44f1-922d-c8d13d4e3a4c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:04.773400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 132/269] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:04.774970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66910-5fa5-44c9-840e-8156d6831163 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.775314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66910-5fa5-44c9-840e-8156d6831163 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.775682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66910-5fa5-44c9-840e-8156d6831163 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:04.777355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 133/270] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:04.784771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06086-24e0-4cc9-b573-0600af8704ef None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.785119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06086-24e0-4cc9-b573-0600af8704ef None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.785455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06086-24e0-4cc9-b573-0600af8704ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:04.806399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-39e06086-24e0-4cc9-b573-0600af8704ef None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:04.807771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-39e06086-24e0-4cc9-b573-0600af8704ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:04.873016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73dd4b-a8e9-44f1-922d-c8d13d4e3a4c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:04.952538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cc08e-8564-4a32-a484-56e9793e0024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:04.953289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cc08e-8564-4a32-a484-56e9793e0024 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.953640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cc08e-8564-4a32-a484-56e9793e0024 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.954007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cc08e-8564-4a32-a484-56e9793e0024 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:04.956001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cc08e-8564-4a32-a484-56e9793e0024 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:04.957574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cc08e-8564-4a32-a484-56e9793e0024 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:04.960107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 134/271] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:04.963799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 138/272] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:04 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:04.965162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09978f-732b-4fca-ae20-ef458ede0c96 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.965452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09978f-732b-4fca-ae20-ef458ede0c96 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.965726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09978f-732b-4fca-ae20-ef458ede0c96 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:04.968516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 135/273] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:04.969853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cc08e-8564-4a32-a484-56e9793e0024 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:04.973926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf7390-8050-4e3c-96d4-044fa316061d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:04.974257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf7390-8050-4e3c-96d4-044fa316061d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:04.974536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf7390-8050-4e3c-96d4-044fa316061d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:05.001903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-91cf7390-8050-4e3c-96d4-044fa316061d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:05.003150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-91cf7390-8050-4e3c-96d4-044fa316061d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:05.013299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:05.059179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:05.126743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 136/274] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:05.128178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf7390-8050-4e3c-96d4-044fa316061d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:05.155457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cc08e-8564-4a32-a484-56e9793e0024 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:05.228828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:05.229374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:05.229648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:05.229915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:05.230680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] RBAC: Adding query filter params (role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=de79363b23ae49c99eab4dc709331b01, user.id=b77355cddbbd4853ad8d59fadfa3b34b) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:05.230986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:05.232438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:05.246078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 139/275] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:04 2022] GET /identity/v3/role_assignments?user.id=b77355cddbbd4853ad8d59fadfa3b34b&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:05.298071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:05.299530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf7390-8050-4e3c-96d4-044fa316061d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:05.367396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a3f43-5fd5-4261-956c-7824284cbfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:05.368064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a3f43-5fd5-4261-956c-7824284cbfca None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:05.368343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a3f43-5fd5-4261-956c-7824284cbfca None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:05.368613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a3f43-5fd5-4261-956c-7824284cbfca None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:05.391676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a3f43-5fd5-4261-956c-7824284cbfca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-2e6a3f43-5fd5-4261-956c-7824284cbfca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:05.397164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:05.398158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:05.400590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 137/276] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:05.404693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:05.405851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a3f43-5fd5-4261-956c-7824284cbfca None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:05.442238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8b443-d4a6-4511-8f80-a3cb1b430156 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:05.458415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:05.495397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f0616-6bf6-4c73-9aac-8165a600afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:05.495975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f0616-6bf6-4c73-9aac-8165a600afcd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:05.496257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f0616-6bf6-4c73-9aac-8165a600afcd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:05.496553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f0616-6bf6-4c73-9aac-8165a600afcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:05.498630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f0616-6bf6-4c73-9aac-8165a600afcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:05.501104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f0616-6bf6-4c73-9aac-8165a600afcd None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:05.503214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 140/277] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:05.503967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f0616-6bf6-4c73-9aac-8165a600afcd None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:05.572234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a3f43-5fd5-4261-956c-7824284cbfca None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:05.623368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:05.624668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:05.624995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:05.625340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:05.626040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:05.626378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:05.628008 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:05.644143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 138/278] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:05.644143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:05.647789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f0616-6bf6-4c73-9aac-8165a600afcd None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:05.689317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38890b18-6c6b-4f7a-afeb-a4a8bff9b026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:05.689971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38890b18-6c6b-4f7a-afeb-a4a8bff9b026 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:05.690281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38890b18-6c6b-4f7a-afeb-a4a8bff9b026 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:05.690555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38890b18-6c6b-4f7a-afeb-a4a8bff9b026 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:05.709068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38890b18-6c6b-4f7a-afeb-a4a8bff9b026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-38890b18-6c6b-4f7a-afeb-a4a8bff9b026 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:05.721139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:05.722079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:05.722925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:05.723538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 141/279] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:05.724610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38890b18-6c6b-4f7a-afeb-a4a8bff9b026 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:05.752599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:05.760350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:05.780674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d73174-03e9-46a7-90ad-dddccc99b2c6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:05.818380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4e54d-a61e-48c6-b445-3d52fbd5df7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:05.819205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e54d-a61e-48c6-b445-3d52fbd5df7e None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:05.826576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e54d-a61e-48c6-b445-3d52fbd5df7e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:05.826576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e54d-a61e-48c6-b445-3d52fbd5df7e None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:05.826576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4e54d-a61e-48c6-b445-3d52fbd5df7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:05.826576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4e54d-a61e-48c6-b445-3d52fbd5df7e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:05.856775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38890b18-6c6b-4f7a-afeb-a4a8bff9b026 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:05.892357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9927294-fa71-48c4-8500-703df1a37526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:05.893004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9927294-fa71-48c4-8500-703df1a37526 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:05.893283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9927294-fa71-48c4-8500-703df1a37526 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:05.893560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9927294-fa71-48c4-8500-703df1a37526 None admin] PATH_INFO: `/v3/users/aeb5941f7b5c49c296518f49970d6c9c` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:05.907572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9927294-fa71-48c4-8500-703df1a37526 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb5941f7b5c49c296518f49970d6c9c)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:05.909066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9927294-fa71-48c4-8500-703df1a37526 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:05.910792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 142/280] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:05 2022] GET /identity/v3/users/aeb5941f7b5c49c296518f49970d6c9c => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:05.911557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9927294-fa71-48c4-8500-703df1a37526 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:05.947313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 139/281] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:33:05 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:33:05.948463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4e54d-a61e-48c6-b445-3d52fbd5df7e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:06.012315 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:06.049879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9927294-fa71-48c4-8500-703df1a37526 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:06.094287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:06.095005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:06.095292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:06.095569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:06.096269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:06.096578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:06.098011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:06.101141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4e54d-a61e-48c6-b445-3d52fbd5df7e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:06.113067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 143/282] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:06.114392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:06.146689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:06.147668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:06.148160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:06.148665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:06.149624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] RBAC: Adding query filter params (role.id=1881f748eade41d6aa06dc2097202f09, user.id=aeb5941f7b5c49c296518f49970d6c9c, scope.system=all) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:06.150053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:06.151653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:06.158416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 140/283] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 8 13:33:05 2022] GET /identity/v3/role_assignments?user.id=aeb5941f7b5c49c296518f49970d6c9c&scope.system=all&role.id=1881f748eade41d6aa06dc2097202f09 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:06.251047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee63112-a4f6-4c2d-a120-06503df2b796 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:06.299435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56387c53-1d27-4ffd-afe4-1240a52ddfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:06.300460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56387c53-1d27-4ffd-afe4-1240a52ddfcc None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:06.300909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56387c53-1d27-4ffd-afe4-1240a52ddfcc None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:06.301395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56387c53-1d27-4ffd-afe4-1240a52ddfcc None admin] PATH_INFO: `/v3/users/b89c4a3ab4f24760a6feaf9d1a367732` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:06.303742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56387c53-1d27-4ffd-afe4-1240a52ddfcc None admin] RBAC: Authorizing `identity:get_user(user_id=b89c4a3ab4f24760a6feaf9d1a367732)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:06.305493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56387c53-1d27-4ffd-afe4-1240a52ddfcc None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:06.307760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 144/284] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:06 2022] GET /identity/v3/users/b89c4a3ab4f24760a6feaf9d1a367732 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:06.313301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:06.332777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:06.442793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b171ec-e486-4f03-a57a-5a90250456bd None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:06.488849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef762a-2d98-446e-850a-8a87f8270896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:06.489515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef762a-2d98-446e-850a-8a87f8270896 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:06.489801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef762a-2d98-446e-850a-8a87f8270896 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:06.490071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef762a-2d98-446e-850a-8a87f8270896 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:06.491300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef762a-2d98-446e-850a-8a87f8270896 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:06.492949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef762a-2d98-446e-850a-8a87f8270896 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:06.495260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 141/285] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:06 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:06.504918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56387c53-1d27-4ffd-afe4-1240a52ddfcc None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:06.508133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:06.602080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8340-5976-4fda-97ac-ec73d8b58ebd None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:06.602457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8340-5976-4fda-97ac-ec73d8b58ebd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:06.602657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8340-5976-4fda-97ac-ec73d8b58ebd None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:06.604250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 142/286] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:06.610531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c42f3-ec3f-4846-a508-92912b0d9267 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:06.610813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c42f3-ec3f-4846-a508-92912b0d9267 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:06.611089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c42f3-ec3f-4846-a508-92912b0d9267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:06.645145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ef4c42f3-ec3f-4846-a508-92912b0d9267 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:06.646463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ef4c42f3-ec3f-4846-a508-92912b0d9267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:06.681103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56387c53-1d27-4ffd-afe4-1240a52ddfcc None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:06.740429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d7333-0950-4fa6-8295-8a8a9a4bcf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:06.741109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d7333-0950-4fa6-8295-8a8a9a4bcf5c None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:06.741393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d7333-0950-4fa6-8295-8a8a9a4bcf5c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:06.741668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d7333-0950-4fa6-8295-8a8a9a4bcf5c None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01/users/b89c4a3ab4f24760a6feaf9d1a367732/roles/44854c86be404c76a5703d725444f549` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:06.747262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d7333-0950-4fa6-8295-8a8a9a4bcf5c None admin] RBAC: Authorizing `identity:create_grant(project_id=de79363b23ae49c99eab4dc709331b01, user_id=b89c4a3ab4f24760a6feaf9d1a367732, role_id=44854c86be404c76a5703d725444f549)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:06.748714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d7333-0950-4fa6-8295-8a8a9a4bcf5c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:06.769322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 143/287] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:06.851385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 145/288] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:33:06 2022] PUT /identity/v3/projects/de79363b23ae49c99eab4dc709331b01/users/b89c4a3ab4f24760a6feaf9d1a367732/roles/44854c86be404c76a5703d725444f549 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:07.132443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c42f3-ec3f-4846-a508-92912b0d9267 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:07.143598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:07.247781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c42f3-ec3f-4846-a508-92912b0d9267 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:07.283873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fef8a-6e10-4fa1-b49f-df543a63bbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:07.284513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fef8a-6e10-4fa1-b49f-df543a63bbe4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:07.284794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fef8a-6e10-4fa1-b49f-df543a63bbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:07.285069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fef8a-6e10-4fa1-b49f-df543a63bbe4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:07.297367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fef8a-6e10-4fa1-b49f-df543a63bbe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-158fef8a-6e10-4fa1-b49f-df543a63bbe4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:07.305702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:07.306690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:07.308227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 144/289] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:07.315601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d7333-0950-4fa6-8295-8a8a9a4bcf5c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:07.413533 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:07.483224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d7333-0950-4fa6-8295-8a8a9a4bcf5c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:07.550075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:07.550754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:07.551040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:07.551315 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:07.552001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] RBAC: Adding query filter params (name=member) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:07.552311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:07.553732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:07.566739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 146/290] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:07.572495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fef8a-6e10-4fa1-b49f-df543a63bbe4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:07.690293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:07.731473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fef8a-6e10-4fa1-b49f-df543a63bbe4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:07.770451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e259-fabe-48de-83f3-2d2988e0b23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:07.771094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e259-fabe-48de-83f3-2d2988e0b23e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:07.771378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e259-fabe-48de-83f3-2d2988e0b23e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:07.771732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e259-fabe-48de-83f3-2d2988e0b23e None admin] PATH_INFO: `/v3/users/d4758c8f613f4e48b00ea834419d963e` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:07.772923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3e259-fabe-48de-83f3-2d2988e0b23e None admin] RBAC: Authorizing `identity:get_user(user_id=d4758c8f613f4e48b00ea834419d963e)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:07.774435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3e259-fabe-48de-83f3-2d2988e0b23e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:07.776458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 145/291] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:07 2022] GET /identity/v3/users/d4758c8f613f4e48b00ea834419d963e => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:07.782062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:07.947165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a256-db4d-4243-99de-15d3da5d613e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:07.980901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:07.996678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b6224-a083-40cf-b579-e5912e102d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:07.997565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b6224-a083-40cf-b579-e5912e102d66 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:07.997673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b6224-a083-40cf-b579-e5912e102d66 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:07.997852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b6224-a083-40cf-b579-e5912e102d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:08.092361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b6224-a083-40cf-b579-e5912e102d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:08.094163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b6224-a083-40cf-b579-e5912e102d66 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:08.225313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 147/292] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:08.231552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e259-fabe-48de-83f3-2d2988e0b23e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:08.369405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e259-fabe-48de-83f3-2d2988e0b23e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:08.420891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2e0df-f08b-4403-8175-9cc8a715a6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:08.422820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2e0df-f08b-4403-8175-9cc8a715a6bd None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:08.423520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2e0df-f08b-4403-8175-9cc8a715a6bd None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:08.424234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2e0df-f08b-4403-8175-9cc8a715a6bd None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:08.425830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2e0df-f08b-4403-8175-9cc8a715a6bd None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:08.428003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2e0df-f08b-4403-8175-9cc8a715a6bd None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:08.429923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 146/293] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:08 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:08.436156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b6224-a083-40cf-b579-e5912e102d66 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:08.493933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:08.598846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b6224-a083-40cf-b579-e5912e102d66 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:08.657542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c451d24-6851-4de7-8bc9-7e19271bf5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:08.658188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c451d24-6851-4de7-8bc9-7e19271bf5d3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:08.658535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c451d24-6851-4de7-8bc9-7e19271bf5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:08.658817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c451d24-6851-4de7-8bc9-7e19271bf5d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:08.659627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c451d24-6851-4de7-8bc9-7e19271bf5d3 None admin] RBAC: Adding query filter params (role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=de79363b23ae49c99eab4dc709331b01, user.id=d4758c8f613f4e48b00ea834419d963e) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:08.659945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c451d24-6851-4de7-8bc9-7e19271bf5d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:08.661402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c451d24-6851-4de7-8bc9-7e19271bf5d3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:08.674255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 148/294] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:08 2022] GET /identity/v3/role_assignments?user.id=d4758c8f613f4e48b00ea834419d963e&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=1881f748eade41d6aa06dc2097202f09 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:08.923819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19c4d3-0276-44dc-b35f-9b5c7618c45d None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:08.924421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19c4d3-0276-44dc-b35f-9b5c7618c45d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:08.924875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19c4d3-0276-44dc-b35f-9b5c7618c45d None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:08.927154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 147/295] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:08.932881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23ee4-ae12-4479-96ec-de435d4731a7 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:08.933164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23ee4-ae12-4479-96ec-de435d4731a7 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:08.933609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23ee4-ae12-4479-96ec-de435d4731a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:08.954998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d1b23ee4-ae12-4479-96ec-de435d4731a7 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:08.956687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d1b23ee4-ae12-4479-96ec-de435d4731a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:09.000429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96fd0-7d15-4d49-94c2-96eefedce1b8 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.001051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96fd0-7d15-4d49-94c2-96eefedce1b8 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.001519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96fd0-7d15-4d49-94c2-96eefedce1b8 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.004628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 148/296] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:33:09.009278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bb32e-402e-4397-9dd6-e9f2cf7fe781 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.009900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bb32e-402e-4397-9dd6-e9f2cf7fe781 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.010426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bb32e-402e-4397-9dd6-e9f2cf7fe781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.018613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:09.030306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-757bb32e-402e-4397-9dd6-e9f2cf7fe781 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:09.031853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-757bb32e-402e-4397-9dd6-e9f2cf7fe781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:09.059468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 149/297] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:09.120972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 149/298] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:09.130549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e8f4e-0f07-430f-b918-ea46bb2014c5 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.130835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e8f4e-0f07-430f-b918-ea46bb2014c5 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.131277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e8f4e-0f07-430f-b918-ea46bb2014c5 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.132788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 150/299] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:09.138985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4c94-088b-46ba-b6dc-bc1cdb586cd7 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.139421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4c94-088b-46ba-b6dc-bc1cdb586cd7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.139789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4c94-088b-46ba-b6dc-bc1cdb586cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.161050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0b7e4c94-088b-46ba-b6dc-bc1cdb586cd7 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:09.162349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0b7e4c94-088b-46ba-b6dc-bc1cdb586cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:09.251478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 150/300] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:09.363861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e8f4e-0f07-430f-b918-ea46bb2014c5 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:09.386812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:09.443876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e4c94-088b-46ba-b6dc-bc1cdb586cd7 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:09.487780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e8f4e-0f07-430f-b918-ea46bb2014c5 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:09.518830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:09.536528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb3387-45dc-4fa3-b3ec-aab38379a984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:09.537462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb3387-45dc-4fa3-b3ec-aab38379a984 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.537890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb3387-45dc-4fa3-b3ec-aab38379a984 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.538356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb3387-45dc-4fa3-b3ec-aab38379a984 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.551340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb3387-45dc-4fa3-b3ec-aab38379a984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c4fb3387-45dc-4fa3-b3ec-aab38379a984 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:09.557628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:09.558781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:09.559597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:09.562211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 151/301] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:09.567507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb3387-45dc-4fa3-b3ec-aab38379a984 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:09.572415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e4c94-088b-46ba-b6dc-bc1cdb586cd7 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:09.613439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaf535-4c3e-4889-b963-d69b189f557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:09.614129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaf535-4c3e-4889-b963-d69b189f557f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.614445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaf535-4c3e-4889-b963-d69b189f557f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.614717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaf535-4c3e-4889-b963-d69b189f557f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.615844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aaf535-4c3e-4889-b963-d69b189f557f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:09.617287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aaf535-4c3e-4889-b963-d69b189f557f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:09.623150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 151/302] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:09.623150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaf535-4c3e-4889-b963-d69b189f557f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:09.665400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:09.691067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb3387-45dc-4fa3-b3ec-aab38379a984 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:09.733767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaf535-4c3e-4889-b963-d69b189f557f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:09.741635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87085e-fa64-43fb-bb23-53570a373e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:09.742417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87085e-fa64-43fb-bb23-53570a373e87 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.742704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87085e-fa64-43fb-bb23-53570a373e87 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.742974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87085e-fa64-43fb-bb23-53570a373e87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.757399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87085e-fa64-43fb-bb23-53570a373e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-8c87085e-fa64-43fb-bb23-53570a373e87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:09.762093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:09.766340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 152/303] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:09.767145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87085e-fa64-43fb-bb23-53570a373e87 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:09.777312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:09.782123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:09.782844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.783133 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.783403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.784190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:09.784496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:09.785861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:09.794183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 152/304] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:09.794855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:09.877678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87085e-fa64-43fb-bb23-53570a373e87 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:09.902195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbcff3-9162-4f46-8c6b-809824494332 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:09.913336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6b464-ea8e-4e29-8e51-1ede23f3cb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:09.913940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b464-ea8e-4e29-8e51-1ede23f3cb73 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.914249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b464-ea8e-4e29-8e51-1ede23f3cb73 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.914519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b464-ea8e-4e29-8e51-1ede23f3cb73 None admin] PATH_INFO: `/v3/groups` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.915136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6b464-ea8e-4e29-8e51-1ede23f3cb73 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:09.916587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6b464-ea8e-4e29-8e51-1ede23f3cb73 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:09.935240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:09.935841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.939749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.939749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.939749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:09.939749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:09.939749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:09.946692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 153/305] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:09.947468 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:09.987396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:09.992071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 153/306] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 8 13:33:09 2022] POST /identity/v3/groups => generated 229 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:33:09.993443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880e2a-c8da-46d9-b2f9-cb0697bac10e None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:09.993728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880e2a-c8da-46d9-b2f9-cb0697bac10e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:09.993999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880e2a-c8da-46d9-b2f9-cb0697bac10e None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:09.995242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 154/307] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:09.995954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24880e2a-c8da-46d9-b2f9-cb0697bac10e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:10.022734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:10.075844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89902fb-4674-4043-a6c5-fd3e78792804 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:10.115049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24880e2a-c8da-46d9-b2f9-cb0697bac10e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:10.118151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca07999-6304-4cc1-a681-01056c9f5c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:10.118817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca07999-6304-4cc1-a681-01056c9f5c49 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:10.119093 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca07999-6304-4cc1-a681-01056c9f5c49 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:10.119364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca07999-6304-4cc1-a681-01056c9f5c49 None admin] PATH_INFO: `/v3/users/aeb5941f7b5c49c296518f49970d6c9c` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:10.120659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca07999-6304-4cc1-a681-01056c9f5c49 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb5941f7b5c49c296518f49970d6c9c)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:10.122300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca07999-6304-4cc1-a681-01056c9f5c49 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:10.124741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 154/308] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:09 2022] GET /identity/v3/users/aeb5941f7b5c49c296518f49970d6c9c => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:10.126025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2cf74-0cd2-4f31-aeff-84b31c1b5cf4 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:10.126360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2cf74-0cd2-4f31-aeff-84b31c1b5cf4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:10.126676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2cf74-0cd2-4f31-aeff-84b31c1b5cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:10.146744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-51e2cf74-0cd2-4f31-aeff-84b31c1b5cf4 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:10.147972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-51e2cf74-0cd2-4f31-aeff-84b31c1b5cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:10.161281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba69af-b528-494f-9fd4-2e2734dae3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:10.161933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba69af-b528-494f-9fd4-2e2734dae3c5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:10.162241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba69af-b528-494f-9fd4-2e2734dae3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:10.162519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba69af-b528-494f-9fd4-2e2734dae3c5 None admin] PATH_INFO: `/v3/users/f1ba68281d0b4c52a7aeacebfd5d41ed` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:10.177530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba69af-b528-494f-9fd4-2e2734dae3c5 None admin] RBAC: Authorizing `identity:get_user(user_id=f1ba68281d0b4c52a7aeacebfd5d41ed)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:10.179029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba69af-b528-494f-9fd4-2e2734dae3c5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:10.180748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 155/309] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:09 2022] GET /identity/v3/users/f1ba68281d0b4c52a7aeacebfd5d41ed => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:10.181528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba69af-b528-494f-9fd4-2e2734dae3c5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:10.230057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 155/310] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:10.231670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2cf74-0cd2-4f31-aeff-84b31c1b5cf4 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:10.254346 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:10.291482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:10.328661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba69af-b528-494f-9fd4-2e2734dae3c5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:10.349713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2cf74-0cd2-4f31-aeff-84b31c1b5cf4 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:10.365773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2ddd-ab2f-4911-a151-d83cb5411817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:10.366473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2ddd-ab2f-4911-a151-d83cb5411817 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:10.367113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2ddd-ab2f-4911-a151-d83cb5411817 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:10.367396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2ddd-ab2f-4911-a151-d83cb5411817 None admin] PATH_INFO: `/v3/system/users/aeb5941f7b5c49c296518f49970d6c9c/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:10.369168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a2ddd-ab2f-4911-a151-d83cb5411817 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aeb5941f7b5c49c296518f49970d6c9c, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:10.370782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a2ddd-ab2f-4911-a151-d83cb5411817 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:10.394792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:10.395405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:10.395684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:10.395952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:10.396691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] RBAC: Adding query filter params (role.id=44854c86be404c76a5703d725444f549, user.id=f1ba68281d0b4c52a7aeacebfd5d41ed, scope.system=all) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:10.396989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:10.398557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:10.410223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 156/311] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 8 13:33:10 2022] GET /identity/v3/role_assignments?user.id=f1ba68281d0b4c52a7aeacebfd5d41ed&scope.system=all&role.id=44854c86be404c76a5703d725444f549 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:10.465840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 156/312] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 8 13:33:10 2022] PUT /identity/v3/system/users/aeb5941f7b5c49c296518f49970d6c9c/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:10.556644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:10.691365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993cb6b-874b-45d9-a997-74863e216868 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:10.711237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:10.729426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73c89-7c0a-45cb-93b8-3e928ab12fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:10.730071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73c89-7c0a-45cb-93b8-3e928ab12fa7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:10.730381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73c89-7c0a-45cb-93b8-3e928ab12fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:10.730656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73c89-7c0a-45cb-93b8-3e928ab12fa7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:10.742379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e73c89-7c0a-45cb-93b8-3e928ab12fa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-e0e73c89-7c0a-45cb-93b8-3e928ab12fa7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:10.747797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:10.748719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:10.750128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 157/313] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:10.755610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2ddd-ab2f-4911-a151-d83cb5411817 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:10.882172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2ddd-ab2f-4911-a151-d83cb5411817 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:10.904011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:10.919365 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:10.919983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:10.920270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:10.920546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:10.921275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:10.921582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:10.923083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:10.933838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 157/314] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:10.940147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73c89-7c0a-45cb-93b8-3e928ab12fa7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:11.052053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73c89-7c0a-45cb-93b8-3e928ab12fa7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:11.087057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313c053-8c67-49b1-a1e9-399a64727a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:11.087686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313c053-8c67-49b1-a1e9-399a64727a5a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:11.087963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313c053-8c67-49b1-a1e9-399a64727a5a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:11.088230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313c053-8c67-49b1-a1e9-399a64727a5a None admin] PATH_INFO: `/v3/users/b89c4a3ab4f24760a6feaf9d1a367732` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:11.089370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313c053-8c67-49b1-a1e9-399a64727a5a None admin] RBAC: Authorizing `identity:get_user(user_id=b89c4a3ab4f24760a6feaf9d1a367732)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:11.090845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313c053-8c67-49b1-a1e9-399a64727a5a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:11.093565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 158/315] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:10 2022] GET /identity/v3/users/b89c4a3ab4f24760a6feaf9d1a367732 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:11.098844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:11.207180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca577c-d45c-4ea1-80e7-83d0977dc839 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:11.245423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c246c6-058a-455d-8a22-5b596ed54309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:11.246040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c246c6-058a-455d-8a22-5b596ed54309 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:11.246349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c246c6-058a-455d-8a22-5b596ed54309 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:11.246626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c246c6-058a-455d-8a22-5b596ed54309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:11.304652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:11.325818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c246c6-058a-455d-8a22-5b596ed54309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:11.327425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c246c6-058a-455d-8a22-5b596ed54309 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:11.442583 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 158/316] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:11.454178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313c053-8c67-49b1-a1e9-399a64727a5a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:11.500315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:11.564844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313c053-8c67-49b1-a1e9-399a64727a5a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:11.601233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd0009-47b1-4422-875c-f360eceeadea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:11.601868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0009-47b1-4422-875c-f360eceeadea None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:11.602158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0009-47b1-4422-875c-f360eceeadea None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:11.602441 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0009-47b1-4422-875c-f360eceeadea None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:11.603558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd0009-47b1-4422-875c-f360eceeadea None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:11.604997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd0009-47b1-4422-875c-f360eceeadea None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:11.606967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 159/317] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:11 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:11.613220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c246c6-058a-455d-8a22-5b596ed54309 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:11.707904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:11.729729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c246c6-058a-455d-8a22-5b596ed54309 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:11.767685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcf046-ecd7-46f4-9697-63b6934690dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:11.768288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcf046-ecd7-46f4-9697-63b6934690dc None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:11.768536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcf046-ecd7-46f4-9697-63b6934690dc None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:11.768747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcf046-ecd7-46f4-9697-63b6934690dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:11.769557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcf046-ecd7-46f4-9697-63b6934690dc None admin] RBAC: Adding query filter params (role.id=44854c86be404c76a5703d725444f549, scope.project.id=de79363b23ae49c99eab4dc709331b01, user.id=b89c4a3ab4f24760a6feaf9d1a367732) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:11.769865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcf046-ecd7-46f4-9697-63b6934690dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:11.771413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcf046-ecd7-46f4-9697-63b6934690dc None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:11.778254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 159/318] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:11 2022] GET /identity/v3/role_assignments?user.id=b89c4a3ab4f24760a6feaf9d1a367732&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=44854c86be404c76a5703d725444f549 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:12.415920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74bc6c-b587-48a0-bd66-18a21ac54d10 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:12.416323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74bc6c-b587-48a0-bd66-18a21ac54d10 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:12.416500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74bc6c-b587-48a0-bd66-18a21ac54d10 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:12.417951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 160/319] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:12.423787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bde44-d629-492d-9e18-acf43381d7c2 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:12.424076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bde44-d629-492d-9e18-acf43381d7c2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:12.424348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bde44-d629-492d-9e18-acf43381d7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:12.443458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-263bde44-d629-492d-9e18-acf43381d7c2 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:12.444669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-263bde44-d629-492d-9e18-acf43381d7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:12.528900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 160/320] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:12.743676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f79a9-9898-464f-839e-942a6ed737d1 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:12.744214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f79a9-9898-464f-839e-942a6ed737d1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:12.744214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f79a9-9898-464f-839e-942a6ed737d1 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:12.745592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 161/321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:12.751281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c85c-b85b-4150-93a2-762c10c2821a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:12.751563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c85c-b85b-4150-93a2-762c10c2821a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:12.751837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c85c-b85b-4150-93a2-762c10c2821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:12.772658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-05e5c85c-b85b-4150-93a2-762c10c2821a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:12.774256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-05e5c85c-b85b-4150-93a2-762c10c2821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:12.803496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f79a9-9898-464f-839e-942a6ed737d1 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:12.848226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 161/322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:12.855807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1298051-e71f-40f8-8170-27227ec01293 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:12.855807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1298051-e71f-40f8-8170-27227ec01293 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:12.855807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1298051-e71f-40f8-8170-27227ec01293 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:12.855807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 162/323] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:12.857021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af9e95-5697-4c85-937a-ce05556b0c83 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:12.857316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af9e95-5697-4c85-937a-ce05556b0c83 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:12.857594 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af9e95-5697-4c85-937a-ce05556b0c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:12.878373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-81af9e95-5697-4c85-937a-ce05556b0c83 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:12.879579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-81af9e95-5697-4c85-937a-ce05556b0c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:12.883133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:12.910349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f79a9-9898-464f-839e-942a6ed737d1 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:12.926159 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:12.946757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264366ca-5d88-4d0c-abf6-8fd7fc8e673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:12.947394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264366ca-5d88-4d0c-abf6-8fd7fc8e673c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:12.947671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264366ca-5d88-4d0c-abf6-8fd7fc8e673c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:12.947940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264366ca-5d88-4d0c-abf6-8fd7fc8e673c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:12.949023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264366ca-5d88-4d0c-abf6-8fd7fc8e673c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:12.949627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 163/324] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:12.950699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264366ca-5d88-4d0c-abf6-8fd7fc8e673c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:12.954173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 162/325] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:12.958244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af9e95-5697-4c85-937a-ce05556b0c83 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.059345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af9e95-5697-4c85-937a-ce05556b0c83 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:13.093753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41c860-b49f-4437-a661-c52a05b217d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:13.094601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41c860-b49f-4437-a661-c52a05b217d8 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:13.095023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41c860-b49f-4437-a661-c52a05b217d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:13.095437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41c860-b49f-4437-a661-c52a05b217d8 None admin] PATH_INFO: `/v3/groups` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:13.096231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41c860-b49f-4437-a661-c52a05b217d8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:13.097806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41c860-b49f-4437-a661-c52a05b217d8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:13.102815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:13.115298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264366ca-5d88-4d0c-abf6-8fd7fc8e673c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.213408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264366ca-5d88-4d0c-abf6-8fd7fc8e673c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:13.214802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 164/326] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 8 13:33:12 2022] POST /identity/v3/groups => generated 226 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:33:13.223189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41c860-b49f-4437-a661-c52a05b217d8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.255530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed636-9cdf-4561-a62e-3cab2033da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:13.256459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed636-9cdf-4561-a62e-3cab2033da10 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:13.256746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed636-9cdf-4561-a62e-3cab2033da10 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:13.257026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed636-9cdf-4561-a62e-3cab2033da10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:13.270568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fed636-9cdf-4561-a62e-3cab2033da10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-a4fed636-9cdf-4561-a62e-3cab2033da10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:13.276992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:13.277968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:13.279370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 163/327] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:33:13.286993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed636-9cdf-4561-a62e-3cab2033da10 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.344144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41c860-b49f-4437-a661-c52a05b217d8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:13.346244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:13.385358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8abbfe-d24f-405f-bd8b-61895f3140b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:13.386072 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8abbfe-d24f-405f-bd8b-61895f3140b2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:13.386383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8abbfe-d24f-405f-bd8b-61895f3140b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:13.386654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8abbfe-d24f-405f-bd8b-61895f3140b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:13.399304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8abbfe-d24f-405f-bd8b-61895f3140b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-6a8abbfe-d24f-405f-bd8b-61895f3140b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:13.404925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:13.405890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:13.408367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 165/328] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:33:13.412462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8abbfe-d24f-405f-bd8b-61895f3140b2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.433282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed636-9cdf-4561-a62e-3cab2033da10 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:13.479466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:13.480335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:13.480617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:13.480897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:13.481568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:13.481874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:13.483374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:13.487182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:13.494717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 164/329] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:13.499905 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.543749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8abbfe-d24f-405f-bd8b-61895f3140b2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:13.599091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:13.599655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:13.599914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:13.600187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:13.600922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] RBAC: Adding query filter params (name=member) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:13.601171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:13.602677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:13.615510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 166/330] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:13.620980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.663446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60989a-bb89-43f9-8dd0-512b49a615b3 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:13.667394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:13.710318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd7673-73dd-4bf3-8102-125761658d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:13.711085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd7673-73dd-4bf3-8102-125761658d4f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:13.711378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd7673-73dd-4bf3-8102-125761658d4f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:13.711657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd7673-73dd-4bf3-8102-125761658d4f None admin] PATH_INFO: `/v3/users/f1ba68281d0b4c52a7aeacebfd5d41ed` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:13.712880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd7673-73dd-4bf3-8102-125761658d4f None admin] RBAC: Authorizing `identity:get_user(user_id=f1ba68281d0b4c52a7aeacebfd5d41ed)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:13.716509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd7673-73dd-4bf3-8102-125761658d4f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:13.716509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 165/331] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:13 2022] GET /identity/v3/users/f1ba68281d0b4c52a7aeacebfd5d41ed => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:13.724241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd7673-73dd-4bf3-8102-125761658d4f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.773609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d76ee-0bd4-4650-9d1a-e87673768da8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:13.799870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:13.820528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1cb03-02af-435c-aeb4-434063903c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:13.821247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1cb03-02af-435c-aeb4-434063903c0a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:13.821526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1cb03-02af-435c-aeb4-434063903c0a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:13.821796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1cb03-02af-435c-aeb4-434063903c0a None admin] PATH_INFO: `/v3/users/aeb5941f7b5c49c296518f49970d6c9c` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:13.823175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1cb03-02af-435c-aeb4-434063903c0a None admin] RBAC: Authorizing `identity:get_user(user_id=aeb5941f7b5c49c296518f49970d6c9c)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:13.824656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1cb03-02af-435c-aeb4-434063903c0a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:13.827089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 167/332] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:13 2022] GET /identity/v3/users/aeb5941f7b5c49c296518f49970d6c9c => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:13.835426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1cb03-02af-435c-aeb4-434063903c0a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:13.880763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd7673-73dd-4bf3-8102-125761658d4f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:13.930111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78aef51-fee2-4e55-aab5-4690aae5505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:13.930816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aef51-fee2-4e55-aab5-4690aae5505f None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:13.931094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aef51-fee2-4e55-aab5-4690aae5505f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:13.931365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aef51-fee2-4e55-aab5-4690aae5505f None admin] PATH_INFO: `/v3/system/users/f1ba68281d0b4c52a7aeacebfd5d41ed/roles/44854c86be404c76a5703d725444f549` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:13.933003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78aef51-fee2-4e55-aab5-4690aae5505f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f1ba68281d0b4c52a7aeacebfd5d41ed, role_id=44854c86be404c76a5703d725444f549)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:13.934493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78aef51-fee2-4e55-aab5-4690aae5505f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:13.970053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1cb03-02af-435c-aeb4-434063903c0a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:14.011843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaecaf-e0ad-40c4-af20-d5ea119ab787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:14.012511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaecaf-e0ad-40c4-af20-d5ea119ab787 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:14.012795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaecaf-e0ad-40c4-af20-d5ea119ab787 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:14.013090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaecaf-e0ad-40c4-af20-d5ea119ab787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:14.013819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baaecaf-e0ad-40c4-af20-d5ea119ab787 None admin] RBAC: Adding query filter params (role.id=1881f748eade41d6aa06dc2097202f09, user.id=aeb5941f7b5c49c296518f49970d6c9c, scope.system=all) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:14.014114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baaecaf-e0ad-40c4-af20-d5ea119ab787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:14.016198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baaecaf-e0ad-40c4-af20-d5ea119ab787 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:14.016521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 166/333] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 8 13:33:13 2022] PUT /identity/v3/system/users/f1ba68281d0b4c52a7aeacebfd5d41ed/roles/44854c86be404c76a5703d725444f549 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:14.034532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 168/334] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 8 13:33:13 2022] GET /identity/v3/role_assignments?user.id=aeb5941f7b5c49c296518f49970d6c9c&scope.system=all&role.id=1881f748eade41d6aa06dc2097202f09 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:16.159887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d38b10-73c1-447c-8fc0-3b4c2fc10c86 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:16.160406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d38b10-73c1-447c-8fc0-3b4c2fc10c86 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:16.160482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d38b10-73c1-447c-8fc0-3b4c2fc10c86 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:16.160930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51955c9-0b97-4b82-866b-62ade44aceda None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:16.161447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51955c9-0b97-4b82-866b-62ade44aceda None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:16.161447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51955c9-0b97-4b82-866b-62ade44aceda None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:16.162736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 167/335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:16.163215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 169/336] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:16.168772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549371a-74e5-46ef-a527-fb253eefa0bf None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:16.169065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549371a-74e5-46ef-a527-fb253eefa0bf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:16.169347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549371a-74e5-46ef-a527-fb253eefa0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:16.173736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01a6d5-3c32-4f44-b8fc-6eaa31fc049f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:16.174020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01a6d5-3c32-4f44-b8fc-6eaa31fc049f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:16.174334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01a6d5-3c32-4f44-b8fc-6eaa31fc049f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:16.214551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5549371a-74e5-46ef-a527-fb253eefa0bf None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:16.215792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5549371a-74e5-46ef-a527-fb253eefa0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:16.218658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-3b01a6d5-3c32-4f44-b8fc-6eaa31fc049f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:16.219882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3b01a6d5-3c32-4f44-b8fc-6eaa31fc049f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:16.294990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:16.298561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:16.352765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 170/337] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:16.367807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 168/338] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:16.815165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01a6d5-3c32-4f44-b8fc-6eaa31fc049f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:16.940559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01a6d5-3c32-4f44-b8fc-6eaa31fc049f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:16.961489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549371a-74e5-46ef-a527-fb253eefa0bf None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:16.986938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8505ea-1411-4e3f-adb4-3dd2bc468f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:16.987595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8505ea-1411-4e3f-adb4-3dd2bc468f5b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:16.987872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8505ea-1411-4e3f-adb4-3dd2bc468f5b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:16.988141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8505ea-1411-4e3f-adb4-3dd2bc468f5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:17.006005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8505ea-1411-4e3f-adb4-3dd2bc468f5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-8b8505ea-1411-4e3f-adb4-3dd2bc468f5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:17.011094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:17.012056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:17.013310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 171/339] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:33:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:17.022175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8505ea-1411-4e3f-adb4-3dd2bc468f5b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:17.100579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:17.214888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549371a-74e5-46ef-a527-fb253eefa0bf None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:17.242683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8505ea-1411-4e3f-adb4-3dd2bc468f5b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:17.280040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4f3cd-7e45-4153-b08d-b69b7f917b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:17.280689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4f3cd-7e45-4153-b08d-b69b7f917b28 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:17.281553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4f3cd-7e45-4153-b08d-b69b7f917b28 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:17.281826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4f3cd-7e45-4153-b08d-b69b7f917b28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:17.308732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4f3cd-7e45-4153-b08d-b69b7f917b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-d1d4f3cd-7e45-4153-b08d-b69b7f917b28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:17.313982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:17.314926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:17.315715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:17.317057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 169/340] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:17.318894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d057e07-864b-42ac-8ddd-893617545131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:17.319524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057e07-864b-42ac-8ddd-893617545131 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:17.319806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057e07-864b-42ac-8ddd-893617545131 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:17.320074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057e07-864b-42ac-8ddd-893617545131 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:17.320769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057e07-864b-42ac-8ddd-893617545131 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:17.321076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057e07-864b-42ac-8ddd-893617545131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:17.322680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057e07-864b-42ac-8ddd-893617545131 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:17.325302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d51e9-ba67-47ff-b9c3-c8ce6b3a17d7 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:17.325586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d51e9-ba67-47ff-b9c3-c8ce6b3a17d7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:17.325862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d51e9-ba67-47ff-b9c3-c8ce6b3a17d7 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:17.327261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 170/341] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:17.328542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d51e9-ba67-47ff-b9c3-c8ce6b3a17d7 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:17.345355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 172/342] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:17.346712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01834-97ff-4163-85d0-7cc5ac8b5f55 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:17.346995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01834-97ff-4163-85d0-7cc5ac8b5f55 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:17.347263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01834-97ff-4163-85d0-7cc5ac8b5f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:17.385902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-cfd01834-97ff-4163-85d0-7cc5ac8b5f55 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:17.387135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cfd01834-97ff-4163-85d0-7cc5ac8b5f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:17.400358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:17.550663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 173/343] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:17.551454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01834-97ff-4163-85d0-7cc5ac8b5f55 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:17.560449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d51e9-ba67-47ff-b9c3-c8ce6b3a17d7 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:17.609727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:17.624373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:17.625606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:17.625925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:17.626428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:17.627353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:17.627662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:17.629086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:17.646879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 171/344] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:17.707231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:17.754306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01834-97ff-4163-85d0-7cc5ac8b5f55 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:17.819008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04c06b-5def-4534-8b99-2837f6fdc1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:17.819564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c06b-5def-4534-8b99-2837f6fdc1c3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:17.819844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c06b-5def-4534-8b99-2837f6fdc1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:17.820081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:17.820194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c06b-5def-4534-8b99-2837f6fdc1c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:17.839551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dedf4-6144-4552-8450-6539fa014956 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:17.903885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04c06b-5def-4534-8b99-2837f6fdc1c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:17.905419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04c06b-5def-4534-8b99-2837f6fdc1c3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:17.919391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbc336-81ab-45f7-93a2-19d36cc4ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:17.920744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbc336-81ab-45f7-93a2-19d36cc4ebef None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:17.921485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbc336-81ab-45f7-93a2-19d36cc4ebef None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:17.921961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbc336-81ab-45f7-93a2-19d36cc4ebef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:17.950534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:18.030742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbc336-81ab-45f7-93a2-19d36cc4ebef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:18.033432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbc336-81ab-45f7-93a2-19d36cc4ebef None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:18.064820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 174/345] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:18.067023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747b28c-989f-4fca-80ef-fc60d0b00f3e None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.067495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747b28c-989f-4fca-80ef-fc60d0b00f3e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.067958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747b28c-989f-4fca-80ef-fc60d0b00f3e None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.071592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 175/346] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:18.072349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747b28c-989f-4fca-80ef-fc60d0b00f3e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:18.147028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:18.164453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 172/347] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:18.165571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbc336-81ab-45f7-93a2-19d36cc4ebef None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:18.251023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747b28c-989f-4fca-80ef-fc60d0b00f3e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:18.292612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:18.302599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f9dad-bc20-4b5c-8423-349f395dd04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:18.303258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f9dad-bc20-4b5c-8423-349f395dd04e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.303540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f9dad-bc20-4b5c-8423-349f395dd04e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.303803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f9dad-bc20-4b5c-8423-349f395dd04e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.309581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbc336-81ab-45f7-93a2-19d36cc4ebef None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:18.317882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f9dad-bc20-4b5c-8423-349f395dd04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-5d1f9dad-bc20-4b5c-8423-349f395dd04e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:18.324760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:18.326535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 176/348] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:18.328698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285f852-7b9d-45d5-96ec-995bc3626b4b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.330090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285f852-7b9d-45d5-96ec-995bc3626b4b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.330090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285f852-7b9d-45d5-96ec-995bc3626b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.357235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d285f852-7b9d-45d5-96ec-995bc3626b4b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:18.358847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d285f852-7b9d-45d5-96ec-995bc3626b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:18.360912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb773c9-bcbd-4de9-b358-59bf46c5881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:18.361891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb773c9-bcbd-4de9-b358-59bf46c5881b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.362207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb773c9-bcbd-4de9-b358-59bf46c5881b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.362473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb773c9-bcbd-4de9-b358-59bf46c5881b None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.364768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb773c9-bcbd-4de9-b358-59bf46c5881b None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:18.366310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb773c9-bcbd-4de9-b358-59bf46c5881b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:18.368050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 173/349] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:18 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:18.368864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb773c9-bcbd-4de9-b358-59bf46c5881b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:18.449534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:18.498587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 177/350] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:18.501094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285f852-7b9d-45d5-96ec-995bc3626b4b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:18.516870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb773c9-bcbd-4de9-b358-59bf46c5881b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:18.576404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662abd7-9660-4c36-820d-6e934361c439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:18.577353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662abd7-9660-4c36-820d-6e934361c439 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.577930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662abd7-9660-4c36-820d-6e934361c439 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.578457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662abd7-9660-4c36-820d-6e934361c439 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.580499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662abd7-9660-4c36-820d-6e934361c439 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:18.583188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662abd7-9660-4c36-820d-6e934361c439 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:18.590316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 174/351] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:18 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:18.590316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662abd7-9660-4c36-820d-6e934361c439 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:18.652382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285f852-7b9d-45d5-96ec-995bc3626b4b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:18.679398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:18.711770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826aa3e-c95a-4465-9117-84527462f644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:18.712774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826aa3e-c95a-4465-9117-84527462f644 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.713255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826aa3e-c95a-4465-9117-84527462f644 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.713744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826aa3e-c95a-4465-9117-84527462f644 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.714671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826aa3e-c95a-4465-9117-84527462f644 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:18.715333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826aa3e-c95a-4465-9117-84527462f644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:18.716984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826aa3e-c95a-4465-9117-84527462f644 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:18.732177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 178/352] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:18.733786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c0bd-da69-4e28-aa57-fdb238e0b8ca None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.734443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c0bd-da69-4e28-aa57-fdb238e0b8ca None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.734919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c0bd-da69-4e28-aa57-fdb238e0b8ca None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.736437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 179/353] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:18.742398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246ea50-0207-4d31-a48c-3b240bcbc0ca None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.742963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246ea50-0207-4d31-a48c-3b240bcbc0ca None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.743245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246ea50-0207-4d31-a48c-3b240bcbc0ca None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.743989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662abd7-9660-4c36-820d-6e934361c439 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:18.744597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 180/354] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:18.746719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246ea50-0207-4d31-a48c-3b240bcbc0ca None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:18.777697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:18.795106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0b94-2d0e-4433-804e-ae60239e632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:18.795789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0b94-2d0e-4433-804e-ae60239e632f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.796067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0b94-2d0e-4433-804e-ae60239e632f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.796339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0b94-2d0e-4433-804e-ae60239e632f None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.804302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e0b94-2d0e-4433-804e-ae60239e632f None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:18.805760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e0b94-2d0e-4433-804e-ae60239e632f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:18.808836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 175/355] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:18 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:18.808836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0b94-2d0e-4433-804e-ae60239e632f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:18.907913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246ea50-0207-4d31-a48c-3b240bcbc0ca None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:18.956931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0b94-2d0e-4433-804e-ae60239e632f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:18.969699 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f3632-2ac2-4575-8fb8-4cb46d24d2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:18.971045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f3632-2ac2-4575-8fb8-4cb46d24d2d7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.971326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f3632-2ac2-4575-8fb8-4cb46d24d2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.971604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f3632-2ac2-4575-8fb8-4cb46d24d2d7 None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:18.972800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f3632-2ac2-4575-8fb8-4cb46d24d2d7 None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:18.974307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f3632-2ac2-4575-8fb8-4cb46d24d2d7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:18.976226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 181/356] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:18 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:18.977538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc20f-37bc-4ae2-8b3d-2d881acb9d3c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:18.977538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc20f-37bc-4ae2-8b3d-2d881acb9d3c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:18.977748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc20f-37bc-4ae2-8b3d-2d881acb9d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.001487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-4edbc20f-37bc-4ae2-8b3d-2d881acb9d3c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:19.003318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4edbc20f-37bc-4ae2-8b3d-2d881acb9d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:19.008749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095a42e-95fe-46d0-8d01-45ae72ced1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:19.009499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095a42e-95fe-46d0-8d01-45ae72ced1f1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.009787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095a42e-95fe-46d0-8d01-45ae72ced1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.010064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095a42e-95fe-46d0-8d01-45ae72ced1f1 None admin] PATH_INFO: `/v3/users/f1ba68281d0b4c52a7aeacebfd5d41ed` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.012294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095a42e-95fe-46d0-8d01-45ae72ced1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=f1ba68281d0b4c52a7aeacebfd5d41ed)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:19.013754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095a42e-95fe-46d0-8d01-45ae72ced1f1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:19.020815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 176/357] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:33:18 2022] GET /identity/v3/users/f1ba68281d0b4c52a7aeacebfd5d41ed => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:19.020815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e809-402e-486c-8b1b-44beb4f1af52 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.020815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e809-402e-486c-8b1b-44beb4f1af52 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.020815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e809-402e-486c-8b1b-44beb4f1af52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.045223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0398e809-402e-486c-8b1b-44beb4f1af52 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:19.046940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0398e809-402e-486c-8b1b-44beb4f1af52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:19.064430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:19.098524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 182/358] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:19.099386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbc20f-37bc-4ae2-8b3d-2d881acb9d3c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.119949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:19.140778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 177/359] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:19.141836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e809-402e-486c-8b1b-44beb4f1af52 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.230696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbc20f-37bc-4ae2-8b3d-2d881acb9d3c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:19.269531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:19.270196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.270469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.270745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.271488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] RBAC: Adding query filter params (group.id=801c475038944fc3862a24da7adf0f30, role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:19.271800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:19.273272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:19.280329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 183/360] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/role_assignments?group.id=801c475038944fc3862a24da7adf0f30&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:19.280329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.286706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e809-402e-486c-8b1b-44beb4f1af52 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:19.334281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:19.397554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd37e8-f7d3-47ed-8dbc-fc0073b00ab8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:19.417517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122cf14-b38a-4151-b3c3-4166832c4562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:19.418222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122cf14-b38a-4151-b3c3-4166832c4562 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.418508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122cf14-b38a-4151-b3c3-4166832c4562 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.418776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122cf14-b38a-4151-b3c3-4166832c4562 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.432566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122cf14-b38a-4151-b3c3-4166832c4562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:19.437893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-8122cf14-b38a-4151-b3c3-4166832c4562 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:19.438435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:19.439353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:19.440642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:19.441298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.441298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.441298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.441502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 178/361] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:19.441574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] RBAC: Adding query filter params (group.id=801c475038944fc3862a24da7adf0f30, role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:19.441574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:19.441703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122cf14-b38a-4151-b3c3-4166832c4562 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.442130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:19.453564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 184/362] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/role_assignments?group.id=801c475038944fc3862a24da7adf0f30&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=1881f748eade41d6aa06dc2097202f09 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:19.454478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.580325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a872-3684-488e-ac87-699a86379f66 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:19.582556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:19.596733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:19.635508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e0f64-e186-4d1a-b46b-cd923d4ed446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:19.636695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0f64-e186-4d1a-b46b-cd923d4ed446 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.637133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0f64-e186-4d1a-b46b-cd923d4ed446 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.638163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0f64-e186-4d1a-b46b-cd923d4ed446 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.660701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e0f64-e186-4d1a-b46b-cd923d4ed446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-2e9e0f64-e186-4d1a-b46b-cd923d4ed446 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:19.666518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:19.667528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:19.668353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:19.668986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122cf14-b38a-4151-b3c3-4166832c4562 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:19.670402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 185/363] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:19.676916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e0f64-e186-4d1a-b46b-cd923d4ed446 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.725649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:19.726307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.726589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.726860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.727602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] RBAC: Adding query filter params (role.id=44854c86be404c76a5703d725444f549, user.id=f1ba68281d0b4c52a7aeacebfd5d41ed, scope.system=all) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:19.727914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:19.729368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:19.748112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 179/364] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/role_assignments?user.id=f1ba68281d0b4c52a7aeacebfd5d41ed&scope.system=all&role.id=44854c86be404c76a5703d725444f549 => generated 510 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:19.749420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.840467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e0f64-e186-4d1a-b46b-cd923d4ed446 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:19.900512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce39534-6127-4de5-acb8-523484a944c7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:19.904044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:19.919760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:19.920464 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.920660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.920931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.921658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] RBAC: Adding query filter params (name=member) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:19.921990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:19.923512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:19.941572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679377-4b07-4f3a-8de0-b56e8fabff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:19.942486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 186/365] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:19.942784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679377-4b07-4f3a-8de0-b56e8fabff09 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:19.943246 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679377-4b07-4f3a-8de0-b56e8fabff09 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:19.943364 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.943964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679377-4b07-4f3a-8de0-b56e8fabff09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:19.957724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d679377-4b07-4f3a-8de0-b56e8fabff09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-9d679377-4b07-4f3a-8de0-b56e8fabff09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:19.963385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:19.964336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:19.965182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:19.968021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 180/366] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:19.969528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679377-4b07-4f3a-8de0-b56e8fabff09 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:19.995164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:20.099681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ba0de-38c5-48df-b0fa-a2baf75395fb None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:20.100622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679377-4b07-4f3a-8de0-b56e8fabff09 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:20.145237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215725f-105d-47c1-948b-0d0a4acfa50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:20.145461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:20.145932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:20.146249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:20.146520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:20.147035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215725f-105d-47c1-948b-0d0a4acfa50a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:20.147221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:20.147483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:20.147887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215725f-105d-47c1-948b-0d0a4acfa50a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:20.148361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215725f-105d-47c1-948b-0d0a4acfa50a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:20.148861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:20.164631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 187/367] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:20.165370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:20.193060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:20.251062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215725f-105d-47c1-948b-0d0a4acfa50a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:20.252586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215725f-105d-47c1-948b-0d0a4acfa50a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:20.296462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:20.320865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4c3eb-65bf-4729-a242-1634d2f5ee3c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:20.371008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:20.371551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:20.371833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:20.372111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:20.372803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:20.373111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:20.375335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:20.384350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 188/368] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:20.387077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:20.392692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 181/369] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:20.396934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215725f-105d-47c1-948b-0d0a4acfa50a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:20.467492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:20.513136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f51c15-2486-41e2-bb98-79419e1cf4e8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:20.525781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215725f-105d-47c1-948b-0d0a4acfa50a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:20.561991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ef284-ec13-4638-9612-edae8a510e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:20.563720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ef284-ec13-4638-9612-edae8a510e19 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:20.564289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ef284-ec13-4638-9612-edae8a510e19 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:20.564629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ef284-ec13-4638-9612-edae8a510e19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:20.575820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eff13-aabc-4d26-963e-8a3614b6abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:20.577016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eff13-aabc-4d26-963e-8a3614b6abf9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:20.577301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eff13-aabc-4d26-963e-8a3614b6abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:20.577579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eff13-aabc-4d26-963e-8a3614b6abf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:20.589250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:20.681069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ef284-ec13-4638-9612-edae8a510e19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:20.684525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ef284-ec13-4638-9612-edae8a510e19 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:20.690909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1eff13-aabc-4d26-963e-8a3614b6abf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:20.692548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1eff13-aabc-4d26-963e-8a3614b6abf9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:20.766568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:20.814737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 189/370] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:20.815623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ef284-ec13-4638-9612-edae8a510e19 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:20.835691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 182/371] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:20.836615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eff13-aabc-4d26-963e-8a3614b6abf9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:20.892109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:20.964255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ef284-ec13-4638-9612-edae8a510e19 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:20.978296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eff13-aabc-4d26-963e-8a3614b6abf9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:21.012755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b58b92-f897-4707-91c2-9d000cd104d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.013494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b58b92-f897-4707-91c2-9d000cd104d7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.013781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b58b92-f897-4707-91c2-9d000cd104d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.014060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b58b92-f897-4707-91c2-9d000cd104d7 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.016301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b58b92-f897-4707-91c2-9d000cd104d7 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.017899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b58b92-f897-4707-91c2-9d000cd104d7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.020140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 190/372] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:20 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.020250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ede10-8be7-42fa-9919-1eb9cfbfcc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.020696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ede10-8be7-42fa-9919-1eb9cfbfcc89 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.020800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b58b92-f897-4707-91c2-9d000cd104d7 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:21.021134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ede10-8be7-42fa-9919-1eb9cfbfcc89 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.021476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ede10-8be7-42fa-9919-1eb9cfbfcc89 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.022717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ede10-8be7-42fa-9919-1eb9cfbfcc89 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.025221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ede10-8be7-42fa-9919-1eb9cfbfcc89 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.027187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 183/373] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:20 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.028014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ede10-8be7-42fa-9919-1eb9cfbfcc89 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:21.078762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:21.144359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b58b92-f897-4707-91c2-9d000cd104d7 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:21.158489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ede10-8be7-42fa-9919-1eb9cfbfcc89 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:21.163569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:21.180682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d6aec-17bb-4c40-a505-9afb07e5473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.181316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d6aec-17bb-4c40-a505-9afb07e5473d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.181596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d6aec-17bb-4c40-a505-9afb07e5473d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.181866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d6aec-17bb-4c40-a505-9afb07e5473d None admin] PATH_INFO: `/v3/projects/273637bb45fb4aad868a7d2db9ada393` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.183027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d6aec-17bb-4c40-a505-9afb07e5473d None admin] RBAC: Authorizing `identity:get_project(project_id=273637bb45fb4aad868a7d2db9ada393)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.184480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d6aec-17bb-4c40-a505-9afb07e5473d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.186514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 191/374] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:21 2022] GET /identity/v3/projects/273637bb45fb4aad868a7d2db9ada393 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.187336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d6aec-17bb-4c40-a505-9afb07e5473d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:21.197385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6c800-ce2f-4908-8ebb-6c109dafd4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.198480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c800-ce2f-4908-8ebb-6c109dafd4dc None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.198967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c800-ce2f-4908-8ebb-6c109dafd4dc None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.199467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c800-ce2f-4908-8ebb-6c109dafd4dc None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.201514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6c800-ce2f-4908-8ebb-6c109dafd4dc None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.205129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6c800-ce2f-4908-8ebb-6c109dafd4dc None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.207581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 184/375] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:21 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.208566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6c800-ce2f-4908-8ebb-6c109dafd4dc None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:21.298734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d6aec-17bb-4c40-a505-9afb07e5473d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:21.339688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed0f9-552c-4e1c-9e7e-cac37c1a6ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.340330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed0f9-552c-4e1c-9e7e-cac37c1a6ae6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.340605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed0f9-552c-4e1c-9e7e-cac37c1a6ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.340875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed0f9-552c-4e1c-9e7e-cac37c1a6ae6 None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.341881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6c800-ce2f-4908-8ebb-6c109dafd4dc None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:21.342593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ed0f9-552c-4e1c-9e7e-cac37c1a6ae6 None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.344489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ed0f9-552c-4e1c-9e7e-cac37c1a6ae6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.346124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 192/376] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:21 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.346738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:21.346926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed0f9-552c-4e1c-9e7e-cac37c1a6ae6 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:21.387292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534da361-9faa-4b21-a5d6-d2865d5e04dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.388291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534da361-9faa-4b21-a5d6-d2865d5e04dc None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.388766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534da361-9faa-4b21-a5d6-d2865d5e04dc None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.389250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534da361-9faa-4b21-a5d6-d2865d5e04dc None admin] PATH_INFO: `/v3/groups/27cef4689db8404e82b752f972de6363` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.395951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534da361-9faa-4b21-a5d6-d2865d5e04dc None admin] RBAC: Authorizing `identity:get_group(group_id=27cef4689db8404e82b752f972de6363)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.397763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534da361-9faa-4b21-a5d6-d2865d5e04dc None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.400090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 185/377] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:21 2022] GET /identity/v3/groups/27cef4689db8404e82b752f972de6363 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.401177 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534da361-9faa-4b21-a5d6-d2865d5e04dc None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:21.414283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:21.462707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed0f9-552c-4e1c-9e7e-cac37c1a6ae6 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:21.500899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.501099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.501311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.501517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.502441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] RBAC: Adding query filter params (group.id=801c475038944fc3862a24da7adf0f30, role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=de79363b23ae49c99eab4dc709331b01) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:21.502763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.504205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.511141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 193/378] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:21 2022] GET /identity/v3/role_assignments?group.id=801c475038944fc3862a24da7adf0f30&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=1881f748eade41d6aa06dc2097202f09 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.511963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:21.519560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534da361-9faa-4b21-a5d6-d2865d5e04dc None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:21.586263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f5612-27e6-40a4-ba37-b0dfd3957b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.587074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f5612-27e6-40a4-ba37-b0dfd3957b86 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.587362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f5612-27e6-40a4-ba37-b0dfd3957b86 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.587637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f5612-27e6-40a4-ba37-b0dfd3957b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.589934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f5612-27e6-40a4-ba37-b0dfd3957b86 None admin] RBAC: Adding query filter params (group.id=801c475038944fc3862a24da7adf0f30, role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=de79363b23ae49c99eab4dc709331b01) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:21.590294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f5612-27e6-40a4-ba37-b0dfd3957b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.591768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f5612-27e6-40a4-ba37-b0dfd3957b86 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.601889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 186/379] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:21 2022] GET /identity/v3/role_assignments?group.id=801c475038944fc3862a24da7adf0f30&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.665978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cceff-b9b2-4f45-a885-89db077ca931 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:21.701931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:21.738373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea41955-ffee-40f5-a263-e766f778042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:21.739014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea41955-ffee-40f5-a263-e766f778042a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.739299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea41955-ffee-40f5-a263-e766f778042a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.739575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea41955-ffee-40f5-a263-e766f778042a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.740330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea41955-ffee-40f5-a263-e766f778042a None admin] RBAC: Adding query filter params (group.id=27cef4689db8404e82b752f972de6363, role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=273637bb45fb4aad868a7d2db9ada393) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:21.740644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea41955-ffee-40f5-a263-e766f778042a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:21.742109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea41955-ffee-40f5-a263-e766f778042a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:21.755046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 194/380] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:21 2022] GET /identity/v3/role_assignments?group.id=27cef4689db8404e82b752f972de6363&scope.project.id=273637bb45fb4aad868a7d2db9ada393&role.id=97f11873550e41dba0cd095afb1dec00 => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:21.929695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69998194-68d8-4a42-8e0d-d261c09dcce7 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.929989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69998194-68d8-4a42-8e0d-d261c09dcce7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.930199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69998194-68d8-4a42-8e0d-d261c09dcce7 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.931679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 187/381] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:21.937659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bce330-f80c-4f76-8aa2-6e2fe9bfb270 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:21.937986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bce330-f80c-4f76-8aa2-6e2fe9bfb270 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:21.938238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bce330-f80c-4f76-8aa2-6e2fe9bfb270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:21.963430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d7bce330-f80c-4f76-8aa2-6e2fe9bfb270 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:21.964684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d7bce330-f80c-4f76-8aa2-6e2fe9bfb270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:22.058100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 195/382] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:22.312805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef10c41-007c-4b1d-b1ce-e472657c2698 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:22.313477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef10c41-007c-4b1d-b1ce-e472657c2698 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:22.313952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef10c41-007c-4b1d-b1ce-e472657c2698 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:22.315811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 188/383] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:22.320896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31fc45-ee20-4a2e-9889-98e7de3df9e4 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:22.321352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31fc45-ee20-4a2e-9889-98e7de3df9e4 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:22.321441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31fc45-ee20-4a2e-9889-98e7de3df9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:22.342399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-7d31fc45-ee20-4a2e-9889-98e7de3df9e4 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:22.343655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7d31fc45-ee20-4a2e-9889-98e7de3df9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:22.363836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef10c41-007c-4b1d-b1ce-e472657c2698 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:22.406888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:22.460590 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 196/384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:22.521635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef10c41-007c-4b1d-b1ce-e472657c2698 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:22.560152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbf690-d233-45a3-9dac-ffbe5a6ec501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:22.561090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbf690-d233-45a3-9dac-ffbe5a6ec501 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:22.561565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbf690-d233-45a3-9dac-ffbe5a6ec501 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:22.562034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbf690-d233-45a3-9dac-ffbe5a6ec501 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:22.576409 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbf690-d233-45a3-9dac-ffbe5a6ec501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-c2cbf690-d233-45a3-9dac-ffbe5a6ec501 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:22.582124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:22.583033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:22.583828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 189/385] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:33:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:22.589187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31fc45-ee20-4a2e-9889-98e7de3df9e4 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:22.690279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:22.758291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31fc45-ee20-4a2e-9889-98e7de3df9e4 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:22.802131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:22.802775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:22.803052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:22.803325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:22.803993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:22.804294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:22.805727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:22.818013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 197/386] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:22.823732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbf690-d233-45a3-9dac-ffbe5a6ec501 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:22.835285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:22.985696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:23.000377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbf690-d233-45a3-9dac-ffbe5a6ec501 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:23.037157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92155e-a830-4ff3-bbaf-eafe8a9c8e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:23.038220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92155e-a830-4ff3-bbaf-eafe8a9c8e4e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:23.038647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92155e-a830-4ff3-bbaf-eafe8a9c8e4e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:23.039061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92155e-a830-4ff3-bbaf-eafe8a9c8e4e None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:23.040432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92155e-a830-4ff3-bbaf-eafe8a9c8e4e None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:23.042105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92155e-a830-4ff3-bbaf-eafe8a9c8e4e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:23.046679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 190/387] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:22 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:23.050701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92155e-a830-4ff3-bbaf-eafe8a9c8e4e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:23.052038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865f09f-d060-4ef5-a35f-a744a0843930 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:23.058161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:23.142915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a9b22-9fd4-4a8d-be74-2cb24c1bf268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:23.143817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a9b22-9fd4-4a8d-be74-2cb24c1bf268 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:23.144239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a9b22-9fd4-4a8d-be74-2cb24c1bf268 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:23.144650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a9b22-9fd4-4a8d-be74-2cb24c1bf268 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:23.162982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92155e-a830-4ff3-bbaf-eafe8a9c8e4e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:23.175067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a9b22-9fd4-4a8d-be74-2cb24c1bf268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-306a9b22-9fd4-4a8d-be74-2cb24c1bf268 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:23.180404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:23.181482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:23.182503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:23.187674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 198/388] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:23.192910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a9b22-9fd4-4a8d-be74-2cb24c1bf268 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:23.202439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba713b-f6c2-44f9-9d64-7a64662eb71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:23.203333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba713b-f6c2-44f9-9d64-7a64662eb71c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:23.203768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba713b-f6c2-44f9-9d64-7a64662eb71c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:23.204304 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba713b-f6c2-44f9-9d64-7a64662eb71c None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:23.205704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba713b-f6c2-44f9-9d64-7a64662eb71c None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:23.207417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba713b-f6c2-44f9-9d64-7a64662eb71c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:23.214334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 191/389] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:23 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:23.221135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba713b-f6c2-44f9-9d64-7a64662eb71c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:23.258507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:23.334783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba713b-f6c2-44f9-9d64-7a64662eb71c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:23.371315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5601c5-cb63-45a7-9ea0-d2baae012e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:23.372466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5601c5-cb63-45a7-9ea0-d2baae012e75 None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:23.372985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5601c5-cb63-45a7-9ea0-d2baae012e75 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:23.373480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5601c5-cb63-45a7-9ea0-d2baae012e75 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/groups/801c475038944fc3862a24da7adf0f30/roles/4ed4b4b515ca4823abbe45566fe27161` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:23.382851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5601c5-cb63-45a7-9ea0-d2baae012e75 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa3a59b969bb486b9ddb33d6df49e5e2, group_id=801c475038944fc3862a24da7adf0f30, role_id=4ed4b4b515ca4823abbe45566fe27161)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:23.384618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5601c5-cb63-45a7-9ea0-d2baae012e75 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:23.418691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a9b22-9fd4-4a8d-be74-2cb24c1bf268 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:23.455745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:23.456361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:23.456699 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:23.456981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:23.457656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] RBAC: Adding query filter params (name=member) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:23.457959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:23.459486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:23.468546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 199/390] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:23.474552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:23.496762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 192/391] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 13:33:23 2022] PUT /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/groups/801c475038944fc3862a24da7adf0f30/roles/4ed4b4b515ca4823abbe45566fe27161 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:23.506809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:23.587055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8ac0b-46f8-47fe-bf11-d41c392cc13b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:23.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e6994-9329-4ed1-8cd2-7936ce55593f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:23.627112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e6994-9329-4ed1-8cd2-7936ce55593f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:23.627448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e6994-9329-4ed1-8cd2-7936ce55593f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:23.627730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e6994-9329-4ed1-8cd2-7936ce55593f None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:23.628879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e6994-9329-4ed1-8cd2-7936ce55593f None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:23.630355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e6994-9329-4ed1-8cd2-7936ce55593f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:23.632236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 200/392] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:23 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:23.639629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5601c5-cb63-45a7-9ea0-d2baae012e75 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:23.763889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:23.780423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5601c5-cb63-45a7-9ea0-d2baae012e75 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:23.818950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bad2cf-4a01-462d-a280-e22358667c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:23.819595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bad2cf-4a01-462d-a280-e22358667c49 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:23.819878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bad2cf-4a01-462d-a280-e22358667c49 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:23.820157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bad2cf-4a01-462d-a280-e22358667c49 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:23.821305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bad2cf-4a01-462d-a280-e22358667c49 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:23.822784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bad2cf-4a01-462d-a280-e22358667c49 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:23.824736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 193/393] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:23 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:23.830400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e6994-9329-4ed1-8cd2-7936ce55593f None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:23.932489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:23.940388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e6994-9329-4ed1-8cd2-7936ce55593f None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:23.976371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c5ab0-eeaf-4d24-916c-4a5ffcccee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:23.977046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c5ab0-eeaf-4d24-916c-4a5ffcccee5d None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:23.977337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c5ab0-eeaf-4d24-916c-4a5ffcccee5d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:23.977623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c5ab0-eeaf-4d24-916c-4a5ffcccee5d None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/groups/801c475038944fc3862a24da7adf0f30/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:23.979717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c5ab0-eeaf-4d24-916c-4a5ffcccee5d None admin] RBAC: Authorizing `identity:create_grant(project_id=aa3a59b969bb486b9ddb33d6df49e5e2, group_id=801c475038944fc3862a24da7adf0f30, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:23.981350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c5ab0-eeaf-4d24-916c-4a5ffcccee5d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:24.063452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 201/394] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 13:33:23 2022] PUT /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2/groups/801c475038944fc3862a24da7adf0f30/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:24.409423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b0e09-aa3d-4ca8-923c-5c0799e9f668 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:24.409827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b0e09-aa3d-4ca8-923c-5c0799e9f668 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:24.410007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b0e09-aa3d-4ca8-923c-5c0799e9f668 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:24.411672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 194/395] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:24.417699 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294b12-4f79-4f47-a953-0b9e34c99dd4 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:24.418388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294b12-4f79-4f47-a953-0b9e34c99dd4 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:24.418867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294b12-4f79-4f47-a953-0b9e34c99dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:24.440224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1e294b12-4f79-4f47-a953-0b9e34c99dd4 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:24.441772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1e294b12-4f79-4f47-a953-0b9e34c99dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:24.523641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eee24-d2e1-4f24-bfd7-2c3de4550cec None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:24.523873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eee24-d2e1-4f24-bfd7-2c3de4550cec None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:24.524175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eee24-d2e1-4f24-bfd7-2c3de4550cec None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:24.524372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 202/396] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:24.525594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 195/397] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:24.531679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe81df-27c7-442e-ac4b-2e15b25f7eea None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:24.532330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe81df-27c7-442e-ac4b-2e15b25f7eea None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:24.532811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe81df-27c7-442e-ac4b-2e15b25f7eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:24.558714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1bbe81df-27c7-442e-ac4b-2e15b25f7eea None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:24.560338 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1bbe81df-27c7-442e-ac4b-2e15b25f7eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:24.644060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e7b8b-652f-4cae-a249-65ffc8844f53 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:24.644530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e7b8b-652f-4cae-a249-65ffc8844f53 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:24.644605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e7b8b-652f-4cae-a249-65ffc8844f53 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:24.646215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 196/398] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:24.650841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545926-1a27-40ef-a184-6758dae09950 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:24.651125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545926-1a27-40ef-a184-6758dae09950 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:24.651402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545926-1a27-40ef-a184-6758dae09950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:24.665648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 203/399] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:24.693228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-39545926-1a27-40ef-a184-6758dae09950 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:24.694798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-39545926-1a27-40ef-a184-6758dae09950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:24.800241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 197/400] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:24.913177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe81df-27c7-442e-ac4b-2e15b25f7eea None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:24.935298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:24.978248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39545926-1a27-40ef-a184-6758dae09950 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.084125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe81df-27c7-442e-ac4b-2e15b25f7eea None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.118335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39545926-1a27-40ef-a184-6758dae09950 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.122494 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:25.155573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a549f3d-c650-406f-bbc6-8a6ceaa5e612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.156365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a549f3d-c650-406f-bbc6-8a6ceaa5e612 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.156644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3c46f-83d6-44ec-bf95-9f74114ebf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.156765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a549f3d-c650-406f-bbc6-8a6ceaa5e612 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.156949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a549f3d-c650-406f-bbc6-8a6ceaa5e612 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.157280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3c46f-83d6-44ec-bf95-9f74114ebf6d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.157565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3c46f-83d6-44ec-bf95-9f74114ebf6d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.157836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3c46f-83d6-44ec-bf95-9f74114ebf6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.168786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a549f3d-c650-406f-bbc6-8a6ceaa5e612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:25.169101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3c46f-83d6-44ec-bf95-9f74114ebf6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-7a549f3d-c650-406f-bbc6-8a6ceaa5e612 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:25.174626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:25.175726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:25.176937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-54b3c46f-83d6-44ec-bf95-9f74114ebf6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:25.177816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:25.179053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 204/401] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:25.180035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3c46f-83d6-44ec-bf95-9f74114ebf6d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.181958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 198/402] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:33:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:25.188967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a549f3d-c650-406f-bbc6-8a6ceaa5e612 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.294503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:25.309226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a549f3d-c650-406f-bbc6-8a6ceaa5e612 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.316334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3c46f-83d6-44ec-bf95-9f74114ebf6d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.348983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.349717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.350348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.350703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.352072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:25.352591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:25.354387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:25.360447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b578e1-453a-4c6a-b18c-11abb2383099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.361196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b578e1-453a-4c6a-b18c-11abb2383099 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.362070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b578e1-453a-4c6a-b18c-11abb2383099 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.362447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b578e1-453a-4c6a-b18c-11abb2383099 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.365514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 199/403] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:25.366593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.371755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:25.380878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b578e1-453a-4c6a-b18c-11abb2383099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-86b578e1-453a-4c6a-b18c-11abb2383099 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:25.387624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:25.388577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:25.389384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:25.392046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 205/404] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:25.393003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b578e1-453a-4c6a-b18c-11abb2383099 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.489631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541a61-4284-4713-b044-adcb9765c659 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.532477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.533663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.534124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.534597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.535390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b578e1-453a-4c6a-b18c-11abb2383099 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.535973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:25.536452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:25.539196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:25.551343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 200/405] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:25.552539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.579070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b45d58-1ed2-47e4-a47f-61feda2919c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.579725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b45d58-1ed2-47e4-a47f-61feda2919c9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.580010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b45d58-1ed2-47e4-a47f-61feda2919c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.580285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b45d58-1ed2-47e4-a47f-61feda2919c9 None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.584056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b45d58-1ed2-47e4-a47f-61feda2919c9 None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:25.585508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b45d58-1ed2-47e4-a47f-61feda2919c9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:25.591801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 206/406] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:25.591801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b45d58-1ed2-47e4-a47f-61feda2919c9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.591801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:25.648153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:25.697119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154a757-8f1f-4f49-bf93-c9a11095c1cc None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.711670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b45d58-1ed2-47e4-a47f-61feda2919c9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.746828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.747364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.747654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.747928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.748691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:25.748947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:25.750828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67895983-0bec-43fc-bfbc-3fd3028977d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.751519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895983-0bec-43fc-bfbc-3fd3028977d0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.751635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:25.751711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895983-0bec-43fc-bfbc-3fd3028977d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.751949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895983-0bec-43fc-bfbc-3fd3028977d0 None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.760332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67895983-0bec-43fc-bfbc-3fd3028977d0 None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:25.760332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67895983-0bec-43fc-bfbc-3fd3028977d0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:25.760697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 207/407] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:25.761807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 201/408] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:25.761913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67895983-0bec-43fc-bfbc-3fd3028977d0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.761988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.843386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:25.877005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67895983-0bec-43fc-bfbc-3fd3028977d0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.882193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c1ca0-e62e-4caa-9766-1ce66595b27d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:25.913643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:25.915000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b38d6a-1fc7-424f-b6f7-3e2b6a115922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.915929 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38d6a-1fc7-424f-b6f7-3e2b6a115922 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.916347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38d6a-1fc7-424f-b6f7-3e2b6a115922 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.916718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38d6a-1fc7-424f-b6f7-3e2b6a115922 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.918003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b38d6a-1fc7-424f-b6f7-3e2b6a115922 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:25.919568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b38d6a-1fc7-424f-b6f7-3e2b6a115922 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:25.926068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f8359-06c8-40f4-a3a7-c6b135b7a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:25.927108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 208/409] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:25.927108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b38d6a-1fc7-424f-b6f7-3e2b6a115922 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:25.927304 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f8359-06c8-40f4-a3a7-c6b135b7a8dd None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:25.927690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f8359-06c8-40f4-a3a7-c6b135b7a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:25.928105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f8359-06c8-40f4-a3a7-c6b135b7a8dd None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:25.929556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f8359-06c8-40f4-a3a7-c6b135b7a8dd None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:25.931166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f8359-06c8-40f4-a3a7-c6b135b7a8dd None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:25.933335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 202/410] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:25.934301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f8359-06c8-40f4-a3a7-c6b135b7a8dd None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:26.046031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b38d6a-1fc7-424f-b6f7-3e2b6a115922 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:26.047193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f8359-06c8-40f4-a3a7-c6b135b7a8dd None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:26.088778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a7899-e3ae-4472-bea1-3b4a0b516751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:26.089464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a7899-e3ae-4472-bea1-3b4a0b516751 None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.090069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a7899-e3ae-4472-bea1-3b4a0b516751 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.090396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a7899-e3ae-4472-bea1-3b4a0b516751 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01/groups/801c475038944fc3862a24da7adf0f30/roles/4ed4b4b515ca4823abbe45566fe27161` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.093325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a7899-e3ae-4472-bea1-3b4a0b516751 None admin] RBAC: Authorizing `identity:create_grant(project_id=de79363b23ae49c99eab4dc709331b01, group_id=801c475038944fc3862a24da7adf0f30, role_id=4ed4b4b515ca4823abbe45566fe27161)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:26.095469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a7899-e3ae-4472-bea1-3b4a0b516751 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:26.095826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88a8fa-0fda-411a-ac57-061682c8a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:26.096608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88a8fa-0fda-411a-ac57-061682c8a99a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.097050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88a8fa-0fda-411a-ac57-061682c8a99a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.097465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88a8fa-0fda-411a-ac57-061682c8a99a None admin] PATH_INFO: `/v3/groups/27cef4689db8404e82b752f972de6363` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.099623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88a8fa-0fda-411a-ac57-061682c8a99a None admin] RBAC: Authorizing `identity:get_group(group_id=27cef4689db8404e82b752f972de6363)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:26.101425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88a8fa-0fda-411a-ac57-061682c8a99a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:26.103207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 209/411] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:25 2022] GET /identity/v3/groups/27cef4689db8404e82b752f972de6363 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:26.103775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88a8fa-0fda-411a-ac57-061682c8a99a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:26.113004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:26.181997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 203/412] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 13:33:25 2022] PUT /identity/v3/projects/de79363b23ae49c99eab4dc709331b01/groups/801c475038944fc3862a24da7adf0f30/roles/4ed4b4b515ca4823abbe45566fe27161 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:26.183559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a7899-e3ae-4472-bea1-3b4a0b516751 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:26.208765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88a8fa-0fda-411a-ac57-061682c8a99a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:26.249245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b001339-6b14-4404-bbd9-35e88b834c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:26.250575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b001339-6b14-4404-bbd9-35e88b834c39 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.250857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b001339-6b14-4404-bbd9-35e88b834c39 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.251200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b001339-6b14-4404-bbd9-35e88b834c39 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01/groups/801c475038944fc3862a24da7adf0f30/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.253242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b001339-6b14-4404-bbd9-35e88b834c39 None admin] RBAC: Authorizing `identity:create_grant(project_id=de79363b23ae49c99eab4dc709331b01, group_id=801c475038944fc3862a24da7adf0f30, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:26.254899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b001339-6b14-4404-bbd9-35e88b834c39 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:26.269470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:26.319104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a7899-e3ae-4472-bea1-3b4a0b516751 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:26.331506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 210/413] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 13:33:26 2022] PUT /identity/v3/projects/de79363b23ae49c99eab4dc709331b01/groups/801c475038944fc3862a24da7adf0f30/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:26.333735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea461d-22d0-4707-83d9-c57c09e35e09 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.334207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea461d-22d0-4707-83d9-c57c09e35e09 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.334500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea461d-22d0-4707-83d9-c57c09e35e09 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.336534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 211/414] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:26.345255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177945-6fba-4a05-b642-706e826efcab None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.345255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177945-6fba-4a05-b642-706e826efcab None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.345255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177945-6fba-4a05-b642-706e826efcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.384123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ff177945-6fba-4a05-b642-706e826efcab None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:26.385720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ff177945-6fba-4a05-b642-706e826efcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:26.392946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d2449-c52b-4d7b-8276-77daf98fe042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:26.393619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d2449-c52b-4d7b-8276-77daf98fe042 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.394067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d2449-c52b-4d7b-8276-77daf98fe042 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.395480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d2449-c52b-4d7b-8276-77daf98fe042 None admin] PATH_INFO: `/v3/projects/273637bb45fb4aad868a7d2db9ada393` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.396817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d2449-c52b-4d7b-8276-77daf98fe042 None admin] RBAC: Authorizing `identity:get_project(project_id=273637bb45fb4aad868a7d2db9ada393)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:26.398610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d2449-c52b-4d7b-8276-77daf98fe042 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:26.401114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 204/415] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:26 2022] GET /identity/v3/projects/273637bb45fb4aad868a7d2db9ada393 => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:26.406411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d2449-c52b-4d7b-8276-77daf98fe042 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:26.477758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:26.489099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 212/416] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:26.550981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d2449-c52b-4d7b-8276-77daf98fe042 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:26.576262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:26.587176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beab325-8b16-43ef-8072-dd712534e613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:26.588044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab325-8b16-43ef-8072-dd712534e613 None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.588158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab325-8b16-43ef-8072-dd712534e613 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.588416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab325-8b16-43ef-8072-dd712534e613 None admin] PATH_INFO: `/v3/projects/273637bb45fb4aad868a7d2db9ada393/groups/27cef4689db8404e82b752f972de6363/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.590495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beab325-8b16-43ef-8072-dd712534e613 None admin] RBAC: Authorizing `identity:create_grant(project_id=273637bb45fb4aad868a7d2db9ada393, group_id=27cef4689db8404e82b752f972de6363, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:26.592021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beab325-8b16-43ef-8072-dd712534e613 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:26.712101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 205/417] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 13:33:26 2022] PUT /identity/v3/projects/273637bb45fb4aad868a7d2db9ada393/groups/27cef4689db8404e82b752f972de6363/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:26.811173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177945-6fba-4a05-b642-706e826efcab None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:26.840761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca2c1-ef5b-453c-b118-fc40aadc1549 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.841093 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca2c1-ef5b-453c-b118-fc40aadc1549 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.841325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eca2c1-ef5b-453c-b118-fc40aadc1549 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.843045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 206/418] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:33:26.849278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d80d-affb-4b9b-af7f-70082a269ee3 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:26.849571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d80d-affb-4b9b-af7f-70082a269ee3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:26.849897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d80d-affb-4b9b-af7f-70082a269ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:26.885154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e9c8d80d-affb-4b9b-af7f-70082a269ee3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:26.886462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e9c8d80d-affb-4b9b-af7f-70082a269ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:26.942384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177945-6fba-4a05-b642-706e826efcab None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:26.998740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b628a-76aa-45f1-8a7d-6e2b0d885d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:27.000971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b628a-76aa-45f1-8a7d-6e2b0d885d1c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:27.001466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b628a-76aa-45f1-8a7d-6e2b0d885d1c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:27.001933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b628a-76aa-45f1-8a7d-6e2b0d885d1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:27.008757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 207/419] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:27.014672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b628a-76aa-45f1-8a7d-6e2b0d885d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-0f0b628a-76aa-45f1-8a7d-6e2b0d885d1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:27.020406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:27.021329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:27.022152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:27.028779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 213/420] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:33:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:27.030782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8d80d-affb-4b9b-af7f-70082a269ee3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:27.136630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8d80d-affb-4b9b-af7f-70082a269ee3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:27.151787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:27.173070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:27.173681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:27.173958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:27.174257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:27.176369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:27.176762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:27.178172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:27.187126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 208/421] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:27.195245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b628a-76aa-45f1-8a7d-6e2b0d885d1c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:27.266771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:27.330165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b628a-76aa-45f1-8a7d-6e2b0d885d1c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:27.377763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7b8f-36ca-4225-b8b0-4fb331d786e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:27.378410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7b8f-36ca-4225-b8b0-4fb331d786e0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:27.378692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7b8f-36ca-4225-b8b0-4fb331d786e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:27.378969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7b8f-36ca-4225-b8b0-4fb331d786e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:27.424959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:27.490062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7b8f-36ca-4225-b8b0-4fb331d786e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:27.491701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7b8f-36ca-4225-b8b0-4fb331d786e0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:27.581960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:27.606784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ff8f2-c407-49ef-89b7-ac007cf522e5 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:27.635857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 214/422] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:27.643826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7b8f-36ca-4225-b8b0-4fb331d786e0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:27.693503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fc531-000f-43c3-b41d-9cfe868ce1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:27.694068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fc531-000f-43c3-b41d-9cfe868ce1e1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:27.694509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fc531-000f-43c3-b41d-9cfe868ce1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:27.694791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fc531-000f-43c3-b41d-9cfe868ce1e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:27.707549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fc531-000f-43c3-b41d-9cfe868ce1e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-967fc531-000f-43c3-b41d-9cfe868ce1e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:27.713611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:27.714956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:27.715766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 209/423] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:33:27.722073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fc531-000f-43c3-b41d-9cfe868ce1e1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:27.763180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:27.839439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fc531-000f-43c3-b41d-9cfe868ce1e1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:27.870286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7b8f-36ca-4225-b8b0-4fb331d786e0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:27.880327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:27.880948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:27.881227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:27.881563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:27.882266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:27.882623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:27.884103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:27.893836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 210/424] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:27.900171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:27.941086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267ae38-2c0f-4c5a-8518-7727b32c5819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:27.941735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267ae38-2c0f-4c5a-8518-7727b32c5819 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:27.942010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267ae38-2c0f-4c5a-8518-7727b32c5819 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:27.942305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267ae38-2c0f-4c5a-8518-7727b32c5819 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:27.943493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267ae38-2c0f-4c5a-8518-7727b32c5819 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:27.945039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267ae38-2c0f-4c5a-8518-7727b32c5819 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:27.946863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 215/425] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:27 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:27.952875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267ae38-2c0f-4c5a-8518-7727b32c5819 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:28.014624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e5787-9baa-475f-bf89-2641efe8f187 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:28.019421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:28.030042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:28.083754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2d995-c42b-472f-a64e-050b04627e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:28.084378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d995-c42b-472f-a64e-050b04627e49 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:28.084660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d995-c42b-472f-a64e-050b04627e49 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:28.084939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d995-c42b-472f-a64e-050b04627e49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:28.182483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2d995-c42b-472f-a64e-050b04627e49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:28.183950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2d995-c42b-472f-a64e-050b04627e49 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:28.193599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267ae38-2c0f-4c5a-8518-7727b32c5819 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:28.239167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be0316-eaf5-4f90-82d7-9e2f1aee154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:28.239722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be0316-eaf5-4f90-82d7-9e2f1aee154c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:28.240003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be0316-eaf5-4f90-82d7-9e2f1aee154c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:28.240279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be0316-eaf5-4f90-82d7-9e2f1aee154c None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:28.241615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be0316-eaf5-4f90-82d7-9e2f1aee154c None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:28.243479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be0316-eaf5-4f90-82d7-9e2f1aee154c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:28.248223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 216/426] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:27 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:28.252220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be0316-eaf5-4f90-82d7-9e2f1aee154c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:28.370471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:28.377064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 211/427] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:28.382824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2d995-c42b-472f-a64e-050b04627e49 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:28.479512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:28.498207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2d995-c42b-472f-a64e-050b04627e49 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:28.502817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be0316-eaf5-4f90-82d7-9e2f1aee154c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:28.537783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ecef1-a63f-4527-96fa-8ff2b45165e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:28.539136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ecef1-a63f-4527-96fa-8ff2b45165e6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:28.539595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ecef1-a63f-4527-96fa-8ff2b45165e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:28.540064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ecef1-a63f-4527-96fa-8ff2b45165e6 None admin] PATH_INFO: `/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:28.541432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ecef1-a63f-4527-96fa-8ff2b45165e6 None admin] RBAC: Authorizing `identity:get_project(project_id=aa3a59b969bb486b9ddb33d6df49e5e2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:28.543436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ecef1-a63f-4527-96fa-8ff2b45165e6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:28.545670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 212/428] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:28 2022] GET /identity/v3/projects/aa3a59b969bb486b9ddb33d6df49e5e2 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:28.552067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ecef1-a63f-4527-96fa-8ff2b45165e6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:28.585472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c40e4d-2880-4bcc-9c0c-aa74b5b622d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:28.586655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c40e4d-2880-4bcc-9c0c-aa74b5b622d8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:28.588103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c40e4d-2880-4bcc-9c0c-aa74b5b622d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:28.588556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c40e4d-2880-4bcc-9c0c-aa74b5b622d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:28.589536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c40e4d-2880-4bcc-9c0c-aa74b5b622d8 None admin] RBAC: Adding query filter params (group.id=801c475038944fc3862a24da7adf0f30, role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:28.590261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c40e4d-2880-4bcc-9c0c-aa74b5b622d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:28.592745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c40e4d-2880-4bcc-9c0c-aa74b5b622d8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:28.605176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 217/429] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:28 2022] GET /identity/v3/role_assignments?group.id=801c475038944fc3862a24da7adf0f30&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 611 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:28.664912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:28.702963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ecef1-a63f-4527-96fa-8ff2b45165e6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:28.745642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039518-6cfb-43bc-94a9-91ce6bbf50d4 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:28.745994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039518-6cfb-43bc-94a9-91ce6bbf50d4 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:28.746224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039518-6cfb-43bc-94a9-91ce6bbf50d4 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:28.747904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 218/430] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:33:28.758129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b18edd-a882-4fab-9703-24021b7ce373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:28.758799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18edd-a882-4fab-9703-24021b7ce373 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:28.759019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18edd-a882-4fab-9703-24021b7ce373 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:28.759295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b18edd-a882-4fab-9703-24021b7ce373 None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:28.761485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b18edd-a882-4fab-9703-24021b7ce373 None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:28.762438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a784c-bc66-49c7-ace7-346c55877e42 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:28.762727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a784c-bc66-49c7-ace7-346c55877e42 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:28.763003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a784c-bc66-49c7-ace7-346c55877e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:28.763106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b18edd-a882-4fab-9703-24021b7ce373 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:28.765089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 213/431] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:28 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:28.772163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b18edd-a882-4fab-9703-24021b7ce373 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:28.811735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-715a784c-bc66-49c7-ace7-346c55877e42 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:28.813950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-715a784c-bc66-49c7-ace7-346c55877e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:28.927690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 219/432] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:28.936161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b18edd-a882-4fab-9703-24021b7ce373 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:28.973814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa11ee7-74c9-4172-a78e-0bffeaeba7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:28.974566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa11ee7-74c9-4172-a78e-0bffeaeba7fd None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:28.975271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa11ee7-74c9-4172-a78e-0bffeaeba7fd None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:28.975551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa11ee7-74c9-4172-a78e-0bffeaeba7fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:28.976297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa11ee7-74c9-4172-a78e-0bffeaeba7fd None admin] RBAC: Adding query filter params (group.id=801c475038944fc3862a24da7adf0f30, role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:28.976659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa11ee7-74c9-4172-a78e-0bffeaeba7fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:28.978107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa11ee7-74c9-4172-a78e-0bffeaeba7fd None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:28.982476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:28.985143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 214/433] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:28 2022] GET /identity/v3/role_assignments?group.id=801c475038944fc3862a24da7adf0f30&scope.project.id=aa3a59b969bb486b9ddb33d6df49e5e2&role.id=1881f748eade41d6aa06dc2097202f09 => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:29.132207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fd14f-8ec3-4806-be95-e496417e26f2 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.132606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fd14f-8ec3-4806-be95-e496417e26f2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.132790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fd14f-8ec3-4806-be95-e496417e26f2 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.134489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 220/434] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:29.140220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072235-c9ca-45a4-9053-b17b80af9325 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.140505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072235-c9ca-45a4-9053-b17b80af9325 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.140814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072235-c9ca-45a4-9053-b17b80af9325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.161125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-92072235-c9ca-45a4-9053-b17b80af9325 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:29.162395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-92072235-c9ca-45a4-9053-b17b80af9325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:29.214330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fd14f-8ec3-4806-be95-e496417e26f2 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:29.246046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 215/435] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:29.314937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:29.331892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fd14f-8ec3-4806-be95-e496417e26f2 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:29.379918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320985b6-d923-49fe-b90d-3008ad2a4413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:29.380570 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320985b6-d923-49fe-b90d-3008ad2a4413 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.380851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320985b6-d923-49fe-b90d-3008ad2a4413 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.381189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320985b6-d923-49fe-b90d-3008ad2a4413 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.394041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320985b6-d923-49fe-b90d-3008ad2a4413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-320985b6-d923-49fe-b90d-3008ad2a4413 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:29.398894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:29.399799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:29.400622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 221/436] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:33:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:29.406372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072235-c9ca-45a4-9053-b17b80af9325 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:29.491812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0cbcc-03be-44a1-aad2-8dbd686dce84 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.492246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0cbcc-03be-44a1-aad2-8dbd686dce84 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.492610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0cbcc-03be-44a1-aad2-8dbd686dce84 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.494310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 222/437] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:33:29.499851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb64b4-25e2-4ad0-81bd-0b3b8ef19222 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.500708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb64b4-25e2-4ad0-81bd-0b3b8ef19222 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.500991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb64b4-25e2-4ad0-81bd-0b3b8ef19222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.520413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072235-c9ca-45a4-9053-b17b80af9325 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:29.521914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c6fb64b4-25e2-4ad0-81bd-0b3b8ef19222 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:29.523997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c6fb64b4-25e2-4ad0-81bd-0b3b8ef19222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:29.552146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:29.562380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:29.566226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.566226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.566226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.566226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:29.566226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:29.566650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:29.576440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 216/438] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:29.577056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:29.608048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 223/439] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:29.609326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb64b4-25e2-4ad0-81bd-0b3b8ef19222 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:29.702767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68336516-31ab-424b-b195-d968e5f76ec2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:29.714539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:29.744433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd8916-8843-4618-aff9-65a7b7908245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:29.745088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8916-8843-4618-aff9-65a7b7908245 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.745373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8916-8843-4618-aff9-65a7b7908245 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.745651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8916-8843-4618-aff9-65a7b7908245 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.748694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb64b4-25e2-4ad0-81bd-0b3b8ef19222 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:29.757445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd8916-8843-4618-aff9-65a7b7908245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-2ecd8916-8843-4618-aff9-65a7b7908245 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:29.762807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:29.764389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:33:29.765889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:29.768500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 217/440] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:29.771713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd8916-8843-4618-aff9-65a7b7908245 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:29.787088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718e15-2254-497f-a709-9de6d243b853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:29.787698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718e15-2254-497f-a709-9de6d243b853 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.787978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718e15-2254-497f-a709-9de6d243b853 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.788253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718e15-2254-497f-a709-9de6d243b853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.833599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:29.869677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8718e15-2254-497f-a709-9de6d243b853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:29.871176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8718e15-2254-497f-a709-9de6d243b853 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:29.892923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd8916-8843-4618-aff9-65a7b7908245 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:29.927274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:29.927893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:29.928176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:29.928459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:29.929127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] RBAC: Adding query filter params (name=member) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:29.929434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:29.930836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:29.939289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 218/441] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:33:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:29.940470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:29.947687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:29.971842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 224/442] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:29.972603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718e15-2254-497f-a709-9de6d243b853 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.044425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44298770-1964-4d0b-8d85-663584f1b876 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.057662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:30.070488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718e15-2254-497f-a709-9de6d243b853 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.079493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0dcfe-95c7-476c-9947-0ce387af5f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.080200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0dcfe-95c7-476c-9947-0ce387af5f6a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.080543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0dcfe-95c7-476c-9947-0ce387af5f6a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.080817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0dcfe-95c7-476c-9947-0ce387af5f6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.091534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0dcfe-95c7-476c-9947-0ce387af5f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-0bf0dcfe-95c7-476c-9947-0ce387af5f6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:30.097917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:30.099019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 219/443] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:33:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:30.099827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0dcfe-95c7-476c-9947-0ce387af5f6a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.107841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d624-de5e-4bbe-8086-2cf37c733262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.108446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d624-de5e-4bbe-8086-2cf37c733262 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.108730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d624-de5e-4bbe-8086-2cf37c733262 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.109008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d624-de5e-4bbe-8086-2cf37c733262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.153810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:30.176271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72d624-de5e-4bbe-8086-2cf37c733262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.177712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72d624-de5e-4bbe-8086-2cf37c733262 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.199347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0dcfe-95c7-476c-9947-0ce387af5f6a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.232698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc0bd-3077-45cc-b536-63d039bc507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.233320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0bd-3077-45cc-b536-63d039bc507f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.233601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0bd-3077-45cc-b536-63d039bc507f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.233884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0bd-3077-45cc-b536-63d039bc507f None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.235552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc0bd-3077-45cc-b536-63d039bc507f None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.236996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc0bd-3077-45cc-b536-63d039bc507f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.238844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 220/444] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:30.239397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc0bd-3077-45cc-b536-63d039bc507f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.280049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:30.287143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 225/445] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:30.287143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d624-de5e-4bbe-8086-2cf37c733262 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.339851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc0bd-3077-45cc-b536-63d039bc507f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.359143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:30.373874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.374515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.374798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.375100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.376289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:30.376601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.377971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.382020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d624-de5e-4bbe-8086-2cf37c733262 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.385901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 221/446] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:30.386734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.416379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d3925-de2d-40aa-a14b-3d5249fe061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.417004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3925-de2d-40aa-a14b-3d5249fe061e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.417278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3925-de2d-40aa-a14b-3d5249fe061e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.417549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3925-de2d-40aa-a14b-3d5249fe061e None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.418664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d3925-de2d-40aa-a14b-3d5249fe061e None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.420097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d3925-de2d-40aa-a14b-3d5249fe061e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.421837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 226/447] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:30.423130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d3925-de2d-40aa-a14b-3d5249fe061e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.492620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c58794-7ccd-4f23-8f58-1f7664d5fc5c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.494045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:30.520156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d3925-de2d-40aa-a14b-3d5249fe061e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.525375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f9c44-e8b5-4292-9c7c-fbaac0e92c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.526006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9c44-e8b5-4292-9c7c-fbaac0e92c25 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.526328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9c44-e8b5-4292-9c7c-fbaac0e92c25 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.526608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9c44-e8b5-4292-9c7c-fbaac0e92c25 None admin] PATH_INFO: `/v3/projects/de79363b23ae49c99eab4dc709331b01` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.527738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f9c44-e8b5-4292-9c7c-fbaac0e92c25 None admin] RBAC: Authorizing `identity:get_project(project_id=de79363b23ae49c99eab4dc709331b01)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.529227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f9c44-e8b5-4292-9c7c-fbaac0e92c25 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.530875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 222/448] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/projects/de79363b23ae49c99eab4dc709331b01 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:30.534983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f9c44-e8b5-4292-9c7c-fbaac0e92c25 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.556018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c245b5-017c-4784-b5d8-9dd025db8e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.556624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c245b5-017c-4784-b5d8-9dd025db8e7c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.556914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c245b5-017c-4784-b5d8-9dd025db8e7c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.557193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c245b5-017c-4784-b5d8-9dd025db8e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.571420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:30.626827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c245b5-017c-4784-b5d8-9dd025db8e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.628293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c245b5-017c-4784-b5d8-9dd025db8e7c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.640633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f9c44-e8b5-4292-9c7c-fbaac0e92c25 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.674890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.675492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.675770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.676042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.676765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] RBAC: Adding query filter params (group.id=801c475038944fc3862a24da7adf0f30, role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=de79363b23ae49c99eab4dc709331b01) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:30.677068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.678524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.684622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 223/449] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/role_assignments?group.id=801c475038944fc3862a24da7adf0f30&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:30.685226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.742571 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 227/450] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:30.747663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:30.748564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c245b5-017c-4784-b5d8-9dd025db8e7c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.804296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:30.842806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33bf6e-7e79-451b-8605-72affc1b62f6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.872944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c245b5-017c-4784-b5d8-9dd025db8e7c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:30.883167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97269fd-684c-458d-8418-14032bdc80ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.883810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97269fd-684c-458d-8418-14032bdc80ee None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.884086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97269fd-684c-458d-8418-14032bdc80ee None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.884360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97269fd-684c-458d-8418-14032bdc80ee None admin] PATH_INFO: `/v3/groups/801c475038944fc3862a24da7adf0f30` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.885467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97269fd-684c-458d-8418-14032bdc80ee None admin] RBAC: Authorizing `identity:get_group(group_id=801c475038944fc3862a24da7adf0f30)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.886921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97269fd-684c-458d-8418-14032bdc80ee None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.888682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 224/451] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/groups/801c475038944fc3862a24da7adf0f30 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:30.895377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97269fd-684c-458d-8418-14032bdc80ee None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:30.913842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748a477-4474-4993-bda2-e7016dbe1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:30.914510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748a477-4474-4993-bda2-e7016dbe1fad None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:30.914798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748a477-4474-4993-bda2-e7016dbe1fad None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:30.915064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748a477-4474-4993-bda2-e7016dbe1fad None admin] PATH_INFO: `/v3/projects/273637bb45fb4aad868a7d2db9ada393` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:30.916268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748a477-4474-4993-bda2-e7016dbe1fad None admin] RBAC: Authorizing `identity:get_project(project_id=273637bb45fb4aad868a7d2db9ada393)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:30.917714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748a477-4474-4993-bda2-e7016dbe1fad None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:30.919589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 228/452] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/projects/273637bb45fb4aad868a7d2db9ada393 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:30.925684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748a477-4474-4993-bda2-e7016dbe1fad None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:31.003984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97269fd-684c-458d-8418-14032bdc80ee None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:31.005444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:31.034710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748a477-4474-4993-bda2-e7016dbe1fad None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:31.039842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:31.040457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:31.040741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:31.041013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:31.041759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] RBAC: Adding query filter params (group.id=801c475038944fc3862a24da7adf0f30, role.id=1881f748eade41d6aa06dc2097202f09, scope.project.id=de79363b23ae49c99eab4dc709331b01) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:31.042325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:31.044361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:31.049852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:31.051177 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 225/453] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/role_assignments?group.id=801c475038944fc3862a24da7adf0f30&scope.project.id=de79363b23ae49c99eab4dc709331b01&role.id=1881f748eade41d6aa06dc2097202f09 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:31.072714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd4feb-e6dc-49a7-9b3c-14147ecd52cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:31.073472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd4feb-e6dc-49a7-9b3c-14147ecd52cd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:31.073756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd4feb-e6dc-49a7-9b3c-14147ecd52cd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:31.074031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd4feb-e6dc-49a7-9b3c-14147ecd52cd None admin] PATH_INFO: `/v3/groups/27cef4689db8404e82b752f972de6363` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:31.075360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd4feb-e6dc-49a7-9b3c-14147ecd52cd None admin] RBAC: Authorizing `identity:get_group(group_id=27cef4689db8404e82b752f972de6363)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:31.076954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd4feb-e6dc-49a7-9b3c-14147ecd52cd None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:31.078829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 229/454] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 13:33:30 2022] GET /identity/v3/groups/27cef4689db8404e82b752f972de6363 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:31.085166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:31.191732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c73b4-15bb-41ae-b906-53235092e321 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:31.226625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a2d0e-daa6-4aae-9c92-2d1b705d26e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:31.227236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a2d0e-daa6-4aae-9c92-2d1b705d26e5 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:31.227521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a2d0e-daa6-4aae-9c92-2d1b705d26e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:31.227795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a2d0e-daa6-4aae-9c92-2d1b705d26e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:31.228527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a2d0e-daa6-4aae-9c92-2d1b705d26e5 None admin] RBAC: Adding query filter params (group.id=27cef4689db8404e82b752f972de6363, role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=273637bb45fb4aad868a7d2db9ada393) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:31.228840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a2d0e-daa6-4aae-9c92-2d1b705d26e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:31.230315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a2d0e-daa6-4aae-9c92-2d1b705d26e5 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:31.236763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 226/455] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 13:33:31 2022] GET /identity/v3/role_assignments?group.id=27cef4689db8404e82b752f972de6363&scope.project.id=273637bb45fb4aad868a7d2db9ada393&role.id=97f11873550e41dba0cd095afb1dec00 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:34.063074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f814-cfcb-4e0a-8379-3304e6c77387 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.063552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f814-cfcb-4e0a-8379-3304e6c77387 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.063552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f814-cfcb-4e0a-8379-3304e6c77387 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.065170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 230/456] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:34.069999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fb087-fbc3-4237-b167-fa6a5ff8ba55 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.070828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fb087-fbc3-4237-b167-fa6a5ff8ba55 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.071303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fb087-fbc3-4237-b167-fa6a5ff8ba55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.099100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-983fb087-fbc3-4237-b167-fa6a5ff8ba55 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:34.100639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-983fb087-fbc3-4237-b167-fa6a5ff8ba55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:34.129061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:34.191437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 227/457] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:34.345214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee182ac-6cf0-47ef-a42e-4c25c35ee7fe None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.345842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee182ac-6cf0-47ef-a42e-4c25c35ee7fe None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.346348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee182ac-6cf0-47ef-a42e-4c25c35ee7fe None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.347925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 231/458] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:34.353097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f475807-aaa7-4fc4-bd16-34ab147b420e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.353618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f475807-aaa7-4fc4-bd16-34ab147b420e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.354165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f475807-aaa7-4fc4-bd16-34ab147b420e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.359524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36110e-4d86-4098-af7f-2907ea1cb0ab None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.359846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36110e-4d86-4098-af7f-2907ea1cb0ab None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.360124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36110e-4d86-4098-af7f-2907ea1cb0ab None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.361690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 232/459] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:34.366826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e0959-be9d-453b-8110-81d30ce40cc8 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.367136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e0959-be9d-453b-8110-81d30ce40cc8 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.367412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e0959-be9d-453b-8110-81d30ce40cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.395708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a31e0959-be9d-453b-8110-81d30ce40cc8 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:34.396914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a31e0959-be9d-453b-8110-81d30ce40cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:34.399806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3f475807-aaa7-4fc4-bd16-34ab147b420e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:34.400973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3f475807-aaa7-4fc4-bd16-34ab147b420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:34.507330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 228/460] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:34.528524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 233/461] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:34.543513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f475807-aaa7-4fc4-bd16-34ab147b420e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:34.682649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f475807-aaa7-4fc4-bd16-34ab147b420e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:34.698559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:34.729970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cae0f-9b68-43d6-905d-32058fa1e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:34.730662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cae0f-9b68-43d6-905d-32058fa1e44e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.730945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cae0f-9b68-43d6-905d-32058fa1e44e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.731222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cae0f-9b68-43d6-905d-32058fa1e44e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.741876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cae0f-9b68-43d6-905d-32058fa1e44e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:34.743395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cae0f-9b68-43d6-905d-32058fa1e44e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-bb4cae0f-9b68-43d6-905d-32058fa1e44e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:34.765521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:34.767062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:34.767932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:34.767932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:34.767932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:34.771388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 229/462] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:34.777732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e0959-be9d-453b-8110-81d30ce40cc8 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:34.784547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2669d5-7c24-4f0d-acd3-0ba0ba10b2f7 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.784830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2669d5-7c24-4f0d-acd3-0ba0ba10b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.785108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2669d5-7c24-4f0d-acd3-0ba0ba10b2f7 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.786738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 230/463] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:33:34.793801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3775d-118f-4c1b-a6df-4ceb894863c4 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.794085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3775d-118f-4c1b-a6df-4ceb894863c4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.794387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3775d-118f-4c1b-a6df-4ceb894863c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.829078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-11e3775d-118f-4c1b-a6df-4ceb894863c4 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:34.830691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-11e3775d-118f-4c1b-a6df-4ceb894863c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:34.908247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 231/464] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:34.909808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3775d-118f-4c1b-a6df-4ceb894863c4 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:34.941403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e0959-be9d-453b-8110-81d30ce40cc8 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:34.956947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:34.992706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:34.993318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:34.993597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:34.993865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:34.994578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:34.994879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:34.996252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:34.997199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:35.008949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 234/465] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:35.009870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:35.036131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3775d-118f-4c1b-a6df-4ceb894863c4 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:35.080052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:35.080861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.081241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.081618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.095402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:35.097398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:35.119367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:35.120350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:35.121748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:35.121748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:35.121748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:35.121748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 232/466] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:35.121748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:35.123823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9168f9-a7bb-40c7-b7d8-9aa9e28edfe8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:35.161667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:35.162249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.162532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.162805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.174991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:35.176511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:35.194115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:35.195100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:35.196536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:35.196536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:35.196536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:35.196536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 235/467] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:35.196536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:35.222832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:35.262128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5631bd-569d-4c9e-bfab-a9a9c4bbec37 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:35.287623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:35.299982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9e30e-36fb-473f-96ee-ac8253add225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:35.300594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9e30e-36fb-473f-96ee-ac8253add225 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.300882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9e30e-36fb-473f-96ee-ac8253add225 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.301138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9e30e-36fb-473f-96ee-ac8253add225 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.301790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9e30e-36fb-473f-96ee-ac8253add225 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:35.303247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9e30e-36fb-473f-96ee-ac8253add225 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:35.304637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9446b9f-8b28-4f56-97ca-000562da192b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:35.339633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:35.340234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.340514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.340785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.341421 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:35.341723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:35.343121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:35.354999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 236/468] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:35.356542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:35.402314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 233/469] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:33:35 2022] POST /identity/v3/users => generated 246 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:33:35.403226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9e30e-36fb-473f-96ee-ac8253add225 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:35.464014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:35.494516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281a5d3-10f3-4e9d-aee8-4e94dcf87f8d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:35.535809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:35.536817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.537300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.537774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.538684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:35.539189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:35.540805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:35.542394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9e30e-36fb-473f-96ee-ac8253add225 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:35.554787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 237/470] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:35.555966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:35.588573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c20300-0e20-4335-bf47-74e3ab96423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:35.589530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c20300-0e20-4335-bf47-74e3ab96423a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.589530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c20300-0e20-4335-bf47-74e3ab96423a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.589781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c20300-0e20-4335-bf47-74e3ab96423a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.606054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c20300-0e20-4335-bf47-74e3ab96423a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:35.607553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c20300-0e20-4335-bf47-74e3ab96423a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-77c20300-0e20-4335-bf47-74e3ab96423a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:35.622654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:35.623755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:35.624592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:35.624592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:35.624592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:35.624592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 234/471] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:35.625249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb78f-e99b-46f9-9c9d-0703a856353d None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.625579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb78f-e99b-46f9-9c9d-0703a856353d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.625865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb78f-e99b-46f9-9c9d-0703a856353d None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.632069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 235/472] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:35.632069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cb78f-e99b-46f9-9c9d-0703a856353d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:35.665965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:35.684715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f41e55-8ffc-452f-a53d-f2bc1db563d1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:35.725033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31eb6-ab2a-45a2-b966-2c8af76225c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:35.725951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31eb6-ab2a-45a2-b966-2c8af76225c3 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.726258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31eb6-ab2a-45a2-b966-2c8af76225c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.726533 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31eb6-ab2a-45a2-b966-2c8af76225c3 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.727178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e31eb6-ab2a-45a2-b966-2c8af76225c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:35.728672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e31eb6-ab2a-45a2-b966-2c8af76225c3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:35.741346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cb78f-e99b-46f9-9c9d-0703a856353d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:35.776355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d01c7-c890-4036-81f4-49fec0611ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:35.776961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d01c7-c890-4036-81f4-49fec0611ae9 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.777242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d01c7-c890-4036-81f4-49fec0611ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.777515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d01c7-c890-4036-81f4-49fec0611ae9 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.778167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d01c7-c890-4036-81f4-49fec0611ae9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:35.779592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d01c7-c890-4036-81f4-49fec0611ae9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:35.815585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:35.820923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 238/473] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:33:35 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:33:35.821715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31eb6-ab2a-45a2-b966-2c8af76225c3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:35.883879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 236/474] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:33:35 2022] POST /identity/v3/users => generated 248 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:33:35.883879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702c95b-2a3b-40c0-93cd-40f9f60b3a15 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:35.883879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702c95b-2a3b-40c0-93cd-40f9f60b3a15 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:35.883879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702c95b-2a3b-40c0-93cd-40f9f60b3a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:35.906129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2702c95b-2a3b-40c0-93cd-40f9f60b3a15 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:35.909307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2702c95b-2a3b-40c0-93cd-40f9f60b3a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:36.005598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31eb6-ab2a-45a2-b966-2c8af76225c3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:36.027954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 237/475] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:36.066614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:36.067237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:36.067519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:36.067789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:36.068445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:36.068749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:36.070163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:36.084964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 239/476] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:36.090521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702c95b-2a3b-40c0-93cd-40f9f60b3a15 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:36.112480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:36.199402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702c95b-2a3b-40c0-93cd-40f9f60b3a15 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:36.238675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ee31b-e22c-4065-9bcb-5ce414ca1c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:36.239531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ee31b-e22c-4065-9bcb-5ce414ca1c33 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:36.239993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ee31b-e22c-4065-9bcb-5ce414ca1c33 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:36.240414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ee31b-e22c-4065-9bcb-5ce414ca1c33 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:36.241232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ee31b-e22c-4065-9bcb-5ce414ca1c33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:36.242928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ee31b-e22c-4065-9bcb-5ce414ca1c33 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:36.311184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:36.377028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:36.423861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c3c0-7a7d-4f94-af27-cf9433f15ea5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:36.440867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 238/477] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:33:36 2022] POST /identity/v3/users => generated 249 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:33:36.484712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:36.486348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:36.486839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:36.487345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:36.488248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:36.489872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:36.534387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.sql.core [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:33:36.534387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 13:33:36.534387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [parameters: {'id': '904e426365184362a59f97e53eb765bc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 13:33:36.534387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88256) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result.read() Sep 08 13:33:36.556131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return ref.to_dict() Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:33:36.557080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application yield resource Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise self.value Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application session.commit() Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:33:36.558645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:33:36.559559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:33:36.560820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result.read() Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:33:36.561696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [parameters: {'id': '904e426365184362a59f97e53eb765bc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:36.563579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 13:33:36.565718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:36.567731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 240/478] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 13:33:36 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:33:36.567867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ee31b-e22c-4065-9bcb-5ce414ca1c33 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:36.695980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:36.728784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ee31b-e22c-4065-9bcb-5ce414ca1c33 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:36.765017 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddfc68-ab7f-4131-90cb-062f7023ecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:36.765703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfc68-ab7f-4131-90cb-062f7023ecf4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:36.766008 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfc68-ab7f-4131-90cb-062f7023ecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:36.766840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfc68-ab7f-4131-90cb-062f7023ecf4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:36.777509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddfc68-ab7f-4131-90cb-062f7023ecf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-fbddfc68-ab7f-4131-90cb-062f7023ecf4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:36.784561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:36.785474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:36.786345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 239/479] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:33:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:36.793363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:36.919580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbccdf9-9a7f-41fb-af0c-a39504ef5c57 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:36.951368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:36.966186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159190fd-377c-4baa-889a-ffd8ab3a8cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:36.967718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159190fd-377c-4baa-889a-ffd8ab3a8cc6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:36.967718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159190fd-377c-4baa-889a-ffd8ab3a8cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:36.968050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159190fd-377c-4baa-889a-ffd8ab3a8cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:36.968848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159190fd-377c-4baa-889a-ffd8ab3a8cc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:36.969805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159190fd-377c-4baa-889a-ffd8ab3a8cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:36.971560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159190fd-377c-4baa-889a-ffd8ab3a8cc6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:36.983372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 241/480] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:37.770389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c457218-edda-4c57-ab8f-9713e7c3a0e5 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:37.770893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c457218-edda-4c57-ab8f-9713e7c3a0e5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:37.770967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c457218-edda-4c57-ab8f-9713e7c3a0e5 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:37.772510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 240/481] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:37.777839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109ecf-30cf-4581-a2cc-965a984a6115 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:37.778262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109ecf-30cf-4581-a2cc-965a984a6115 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:37.778449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109ecf-30cf-4581-a2cc-965a984a6115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:37.803197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5b109ecf-30cf-4581-a2cc-965a984a6115 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:37.804405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5b109ecf-30cf-4581-a2cc-965a984a6115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:37.904455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 242/482] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:38.290917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c457218-edda-4c57-ab8f-9713e7c3a0e5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:38.511844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c457218-edda-4c57-ab8f-9713e7c3a0e5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:38.544774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b2fcf-c358-4aa4-bf14-9c91a6793912 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:38.544963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b2fcf-c358-4aa4-bf14-9c91a6793912 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:38.545284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b2fcf-c358-4aa4-bf14-9c91a6793912 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:38.548066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 243/483] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:38.552177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2c913-3adb-4c82-91b1-df0c31f1e759 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:38.552590 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2c913-3adb-4c82-91b1-df0c31f1e759 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:38.552976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2c913-3adb-4c82-91b1-df0c31f1e759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:38.559392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c904a-bcc6-4c73-888e-0744743b51c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:38.560031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c904a-bcc6-4c73-888e-0744743b51c1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:38.560311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c904a-bcc6-4c73-888e-0744743b51c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:38.560591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c904a-bcc6-4c73-888e-0744743b51c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:38.563274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:38.572298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-7be2c913-3adb-4c82-91b1-df0c31f1e759 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:38.573514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7be2c913-3adb-4c82-91b1-df0c31f1e759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:38.579578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c904a-bcc6-4c73-888e-0744743b51c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-4b7c904a-bcc6-4c73-888e-0744743b51c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:38.584374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:38.586713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:38.587516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 241/484] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:38.592375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c904a-bcc6-4c73-888e-0744743b51c1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:38.685202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 244/485] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:38.686755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024119-ab10-41b5-9b5b-fcced9d9269d None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:38.687031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024119-ab10-41b5-9b5b-fcced9d9269d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:38.687305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024119-ab10-41b5-9b5b-fcced9d9269d None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:38.694128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 245/486] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:38.696104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903da135-abea-4422-ac9f-d27aca112e89 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:38.696708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903da135-abea-4422-ac9f-d27aca112e89 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:38.698425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903da135-abea-4422-ac9f-d27aca112e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:38.730223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-903da135-abea-4422-ac9f-d27aca112e89 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:38.731761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-903da135-abea-4422-ac9f-d27aca112e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:38.738500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c904a-bcc6-4c73-888e-0744743b51c1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:38.783780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:38.804420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 246/487] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:38.813950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:38.815224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:38.815701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:38.816165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:38.817061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:38.817624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:38.819311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:38.834126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 242/488] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:38.839479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903da135-abea-4422-ac9f-d27aca112e89 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:38.995592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.005198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903da135-abea-4422-ac9f-d27aca112e89 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.050894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.051536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.051819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.052095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.064512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:39.066441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:39.083550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:39.084537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:39.085395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:39.085395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:39.085395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:39.090189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 247/489] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:39.091697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.120200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:39.137864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:39.176002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3185a63-a33c-4d8f-b008-5b118145bd0f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.213416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef125a3-f05d-4ec6-ae18-7373c1ddb4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.214056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef125a3-f05d-4ec6-ae18-7373c1ddb4b2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.214415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef125a3-f05d-4ec6-ae18-7373c1ddb4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.214633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef125a3-f05d-4ec6-ae18-7373c1ddb4b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.216653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdaf69-405d-49d6-a246-40a2e91a14f9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.225831 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef125a3-f05d-4ec6-ae18-7373c1ddb4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-2ef125a3-f05d-4ec6-ae18-7373c1ddb4b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:39.231742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:39.232630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:39.233458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:39.238470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 243/490] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:39.239339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef125a3-f05d-4ec6-ae18-7373c1ddb4b2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.257668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.258605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.259030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.259492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.260378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:39.260926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:39.262539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:39.275915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 248/491] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:39.276919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.370625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef125a3-f05d-4ec6-ae18-7373c1ddb4b2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.390009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:39.398790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:39.413449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7c366-931e-4b1e-b5c1-c7ee6666cc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.414888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c366-931e-4b1e-b5c1-c7ee6666cc3d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.415467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c366-931e-4b1e-b5c1-c7ee6666cc3d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.415983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c366-931e-4b1e-b5c1-c7ee6666cc3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.418096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539816-3ee0-454e-9e7f-0515ac8f3580 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.428744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7c366-931e-4b1e-b5c1-c7ee6666cc3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-d2b7c366-931e-4b1e-b5c1-c7ee6666cc3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:39.435169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:39.436141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:39.436943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 244/492] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:39.438943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7c366-931e-4b1e-b5c1-c7ee6666cc3d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.463842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bccdd0-c677-450c-a4b4-017344e14d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.464602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bccdd0-c677-450c-a4b4-017344e14d12 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.464984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bccdd0-c677-450c-a4b4-017344e14d12 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.465361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bccdd0-c677-450c-a4b4-017344e14d12 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.466147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bccdd0-c677-450c-a4b4-017344e14d12 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:39.466595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bccdd0-c677-450c-a4b4-017344e14d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:39.467979 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bccdd0-c677-450c-a4b4-017344e14d12 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:39.476785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 249/493] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:39.477963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ed73-9525-49fe-8b52-ce52bf5b52ef None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.478394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ed73-9525-49fe-8b52-ce52bf5b52ef None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.478761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ed73-9525-49fe-8b52-ce52bf5b52ef None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.480209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 250/494] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:39.485062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7ed73-9525-49fe-8b52-ce52bf5b52ef None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.552213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7c366-931e-4b1e-b5c1-c7ee6666cc3d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.588110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7ed73-9525-49fe-8b52-ce52bf5b52ef None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.590983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.591794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.592124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.592598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.601508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:39.603023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:39.617076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:39.618234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:33:39.619348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:39.625514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 245/495] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:39.625514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.639088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c25f9e-2dac-4466-9c4b-bf73a53410e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.639677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25f9e-2dac-4466-9c4b-bf73a53410e1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.640024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25f9e-2dac-4466-9c4b-bf73a53410e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.640393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25f9e-2dac-4466-9c4b-bf73a53410e1 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.641101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c25f9e-2dac-4466-9c4b-bf73a53410e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:39.641505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c25f9e-2dac-4466-9c4b-bf73a53410e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:39.643312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c25f9e-2dac-4466-9c4b-bf73a53410e1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:39.651555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 251/496] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:39.656573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f429-5cd4-4ade-895b-e59e516fbafc None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.656573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f429-5cd4-4ade-895b-e59e516fbafc None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.656573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f429-5cd4-4ade-895b-e59e516fbafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.670616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:39.671697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-08c4f429-5cd4-4ade-895b-e59e516fbafc None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:39.673471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-08c4f429-5cd4-4ade-895b-e59e516fbafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:39.676324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:39.746178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 252/497] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:39.746178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f429-5cd4-4ade-895b-e59e516fbafc None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.764773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf9a0d-1ea2-48a0-8352-f8784618a1cb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.803659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.805379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.805903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.806637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.820784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:39.823938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:39.843864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:39.845699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:39.846585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:39.846585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:39.846585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:39.853756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 246/498] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:39.853756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.860482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f429-5cd4-4ade-895b-e59e516fbafc None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:39.900418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:39.900956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:39.901229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:39.901493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:39.902274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:39.902675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:39.904126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:39.931732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:39.948460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 253/499] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:39.948460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:39.985961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.001999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daf132-d667-4a29-90f6-8b328fefdf58 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.041342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.041966 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.042271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.042543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.053383 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.054857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.058496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50fbc6-5771-4254-963b-3ec704ee7917 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:40.068768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:40.069840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:40.075452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:40.075452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:40.075452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:40.075452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 247/500] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:40.075452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.097288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbb331-8a13-4cfd-9380-0bf7a5ca97b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.097895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbb331-8a13-4cfd-9380-0bf7a5ca97b4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.098207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbb331-8a13-4cfd-9380-0bf7a5ca97b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.098491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbb331-8a13-4cfd-9380-0bf7a5ca97b4 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.099137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbb331-8a13-4cfd-9380-0bf7a5ca97b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:40.099438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbb331-8a13-4cfd-9380-0bf7a5ca97b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.100882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbb331-8a13-4cfd-9380-0bf7a5ca97b4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.115247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 254/501] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:40.117323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08dcb4-9c0f-4021-b7a3-50edfbecbbfb None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.117652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08dcb4-9c0f-4021-b7a3-50edfbecbbfb None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.117964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08dcb4-9c0f-4021-b7a3-50edfbecbbfb None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.119403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 255/502] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:40.119938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08dcb4-9c0f-4021-b7a3-50edfbecbbfb None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.178916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8b53f-11c2-4aed-b25f-bd5ba2e7a82a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.179180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.212894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d535d-cdbc-45cb-adfb-cfbb8ce33761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.213498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d535d-cdbc-45cb-adfb-cfbb8ce33761 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.213784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d535d-cdbc-45cb-adfb-cfbb8ce33761 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.214055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d535d-cdbc-45cb-adfb-cfbb8ce33761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.223550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08dcb4-9c0f-4021-b7a3-50edfbecbbfb None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.224635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.256907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c3a59-859f-4517-a798-fa82e82797cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.257529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3a59-859f-4517-a798-fa82e82797cc None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.257810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3a59-859f-4517-a798-fa82e82797cc None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.258084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3a59-859f-4517-a798-fa82e82797cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.268823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c3a59-859f-4517-a798-fa82e82797cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-6f8c3a59-859f-4517-a798-fa82e82797cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:40.274026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:40.274998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:40.275834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 256/503] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:40.276490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c3a59-859f-4517-a798-fa82e82797cc None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.280635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d535d-cdbc-45cb-adfb-cfbb8ce33761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.282188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d535d-cdbc-45cb-adfb-cfbb8ce33761 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.374860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c3a59-859f-4517-a798-fa82e82797cc None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.382160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 248/504] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:40.383845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d535d-cdbc-45cb-adfb-cfbb8ce33761 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.393175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.409471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d793ae5-1c45-44e8-b4f2-45f92d0fc84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.410114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d793ae5-1c45-44e8-b4f2-45f92d0fc84f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.410423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d793ae5-1c45-44e8-b4f2-45f92d0fc84f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.410696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d793ae5-1c45-44e8-b4f2-45f92d0fc84f None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.411337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d793ae5-1c45-44e8-b4f2-45f92d0fc84f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:40.411644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d793ae5-1c45-44e8-b4f2-45f92d0fc84f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.413022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d793ae5-1c45-44e8-b4f2-45f92d0fc84f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.429958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.430114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 257/505] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:40.430114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2635c-03df-4c3d-b8cf-ced6448318ae None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.430114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2635c-03df-4c3d-b8cf-ced6448318ae None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.430114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2635c-03df-4c3d-b8cf-ced6448318ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.446117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-50a2635c-03df-4c3d-b8cf-ced6448318ae None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:40.447348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-50a2635c-03df-4c3d-b8cf-ced6448318ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:40.483009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d535d-cdbc-45cb-adfb-cfbb8ce33761 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.513560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 258/506] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:40.514438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2635c-03df-4c3d-b8cf-ced6448318ae None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.521264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.521896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.522215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.522472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.529666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.531147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:40.542969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:40.543868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 249/507] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:40.544764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.639801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2635c-03df-4c3d-b8cf-ced6448318ae None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.650636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.654052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.656852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64055b-88cf-4cd1-8f45-c8cd3c44b1a7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.678066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.679044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.679329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.679607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.680338 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:40.680596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.682023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.691939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 259/508] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:40.692643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.700594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.701239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.701522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.701846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.712832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.716156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:40.731939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:40.732871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:40.733732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:40.733732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:40.733732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:40.733732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 250/509] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:40.734596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.809900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3162-2bc2-4e51-93e0-baff23577086 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.837982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1d941-0a6c-436d-a24a-794d7500e78d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:40.844157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.844798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.845079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.845618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.853033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.854518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:40.868555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:40.870188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 260/510] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:40.871360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.878276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:40.878891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:40.879171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:40.879446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:40.880175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:40.880482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:40.881922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:40.905167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.929471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 251/511] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:40.930689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:40.943994 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:40.979887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef27f-3731-4d3b-9e79-c9acbb07885b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.015163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.015778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.016655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.016959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.027193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.028718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.033238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6dfb9-94a4-471e-93fb-f432dcc51671 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:41.043024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:41.044092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:41.045161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:41.045161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:41.045161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:41.045161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 261/512] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:41.045161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.073594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.074230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.074510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.074783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.075431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:41.075725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.077099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.089426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 252/513] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:41.090074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.132198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:41.150175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34324a98-5eab-4eff-b2f2-099032117734 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.158125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:41.183339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.183895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.184179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.184491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.187593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019daabb-6638-447a-b5e3-0d3e565e2b0a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.195540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.197045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:41.211427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:41.212379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:41.213238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:41.213238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:41.213238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:41.213238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 262/514] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:41.215012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.222238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.222850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.223129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.223403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.224132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:41.224436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.225868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.242846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 253/515] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:41.243649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.320334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685998-d11c-476b-bcc6-234f4f4cfdd4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.344264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97527-5a90-4209-b29e-1d19e5c451f6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.354128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef4ed3-2833-498a-be90-4c0a3a79d9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.354761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4ed3-2833-498a-be90-4c0a3a79d9c7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.355046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4ed3-2833-498a-be90-4c0a3a79d9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.355317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4ed3-2833-498a-be90-4c0a3a79d9c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.357947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:41.371310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:41.379435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.380047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.380327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.380607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.381249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:41.381550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.382970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.393972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 254/516] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:41.394838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.427492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef4ed3-2833-498a-be90-4c0a3a79d9c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.428982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef4ed3-2833-498a-be90-4c0a3a79d9c7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.492999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb73044-c82b-412b-b13a-7c449dd2d491 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.526297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 263/517] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:41.526460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.527066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.527188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef4ed3-2833-498a-be90-4c0a3a79d9c7 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.527381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.527626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.536346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.537813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:41.550562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:41.551428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 255/518] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:41.552802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.564319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:41.602331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:41.629387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef4ed3-2833-498a-be90-4c0a3a79d9c7 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.653300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c470180-e7f0-4bdb-83a2-95a267fad828 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.664127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.664736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.665021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.665299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.665987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:41.666326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.667704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.683708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 264/519] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:41.683708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.692380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0963b-d0f1-4c20-a740-d0e1875215e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.692988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0963b-d0f1-4c20-a740-d0e1875215e0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.693265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0963b-d0f1-4c20-a740-d0e1875215e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.693539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0963b-d0f1-4c20-a740-d0e1875215e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.761987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0963b-d0f1-4c20-a740-d0e1875215e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.763419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0963b-d0f1-4c20-a740-d0e1875215e0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.773512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:41.780030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c01e6-a579-4232-894d-a0f01a3a20cd None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.809655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:41.813623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.814289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.814584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.814838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.822290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.823739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:41.835592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:41.836809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:41.837722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 265/520] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:41.838313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.866261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 256/521] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:41.867649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0963b-d0f1-4c20-a740-d0e1875215e0 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:41.942063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24c830-3654-4cd3-8ab6-8041dfc8ab9e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.964736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0963b-d0f1-4c20-a740-d0e1875215e0 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:41.976536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.977156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.977434 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.977701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.988318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:41.989873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:41.997623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:41.998247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:41.998521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:41.998790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:41.999471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:41.999769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.001732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:42.004642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:42.005838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:42.006921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:42.006921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:42.006921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:42.006921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 266/522] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:42.006921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.015219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:42.021777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 257/523] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:42.022721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.034975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:42.107269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640320c2-28c8-4b3e-b6a4-6f6efcbc9299 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.122404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749ab-5a14-4618-b947-673a35e9ba54 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.140144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1fbc7-f415-40c7-bc99-15ee42d68b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.141248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1fbc7-f415-40c7-bc99-15ee42d68b8b None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.141541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1fbc7-f415-40c7-bc99-15ee42d68b8b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.141816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1fbc7-f415-40c7-bc99-15ee42d68b8b None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.142499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1fbc7-f415-40c7-bc99-15ee42d68b8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.143941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1fbc7-f415-40c7-bc99-15ee42d68b8b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.158146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.158760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.159040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.159315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.160040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:42.160345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.161776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.178580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 258/524] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:42.179379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.241837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:42.245235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 267/525] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:33:42 2022] POST /identity/v3/users => generated 247 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:33:42.246062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1fbc7-f415-40c7-bc99-15ee42d68b8b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.280620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cde18-e6e4-4615-aa11-a1cf63638137 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.318224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.318942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.319226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.319503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.325715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:42.332326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.333935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:42.352274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:42.353459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:42.354569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:42.354569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:42.354569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:42.355252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 259/526] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:42.356120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.381002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1fbc7-f415-40c7-bc99-15ee42d68b8b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.419049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.419665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.419985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.420292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.421002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:42.421595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.422748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.440207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 268/527] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:42.440207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.478936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4f6c6-5e6d-4753-a08e-f27febfc8f99 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.496009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:42.514749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.515358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.515648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.516511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.517242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=c24d70e571c64f018aa199c4c08d153e) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:42.517546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.518996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.524710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 260/528] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/role_assignments?user.id=c24d70e571c64f018aa199c4c08d153e&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:42.529561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.557153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dbae6-e5f7-4e44-8398-494c11d9a59b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.590365 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:42.607125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11800547-2fff-496d-9a16-1d55561a2f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.608187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11800547-2fff-496d-9a16-1d55561a2f21 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.608646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11800547-2fff-496d-9a16-1d55561a2f21 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.609110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11800547-2fff-496d-9a16-1d55561a2f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.657214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9141-e975-4de6-95b7-9d005f2f5786 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.703252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11800547-2fff-496d-9a16-1d55561a2f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.704774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11800547-2fff-496d-9a16-1d55561a2f21 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.711335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.712314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.712813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.713347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.714858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:42.715425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.717429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.734163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 261/529] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:42.736310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.791822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:42.816842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 269/530] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:42.817860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11800547-2fff-496d-9a16-1d55561a2f21 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.843093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:42.870656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576b7b4-7a8b-49b7-b05c-d3891bc83bfe None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.915485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.916985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.917811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.918318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.928400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.930982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:42.936314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11800547-2fff-496d-9a16-1d55561a2f21 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:42.946477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:42.947395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:42.948248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:42.948248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:42.948248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:42.948248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:42.948248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:42.948248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:42.948248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:42.948248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:42.950908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 262/531] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:42.952351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:42.979429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:42.980159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:42.980543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:42.980920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:42.988733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:42.990252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:43.003700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:43.004796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:43.005875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:43.005875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:43.005875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:43.005875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:43.005875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:43.005875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:43.005875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:43.005875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:43.012292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 270/532] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:43.013271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:43.084312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf107f-9987-458f-b5c1-ed6e0143028a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:43.085771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:43.114593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:43.126960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:43.127910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:43.128420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:43.129049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:43.135234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1dfc9-4242-4df6-89df-8f5215423345 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:43.144506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:43.146770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:43.164318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:43.165249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:43.166152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:43.166152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:43.166152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:43.172249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 263/533] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:43.172249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:43.179972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:43.180589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:43.180873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:43.181151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:43.181860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:43.182181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:43.183759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:43.211660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 271/534] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:43.212580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:43.301183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950e01a-f2fa-4e0f-8294-a71e8440223c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:43.336372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4814d-c3bb-41cc-8c0f-be6d70084ad9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:43.338630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24719139-9de5-404a-891f-498528e21e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:43.339895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24719139-9de5-404a-891f-498528e21e92 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:43.339895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24719139-9de5-404a-891f-498528e21e92 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:43.339895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24719139-9de5-404a-891f-498528e21e92 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:43.340570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24719139-9de5-404a-891f-498528e21e92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:43.340882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24719139-9de5-404a-891f-498528e21e92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:43.342359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24719139-9de5-404a-891f-498528e21e92 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:43.366176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 264/535] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:43.366176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24719139-9de5-404a-891f-498528e21e92 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:43.382427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:43.393222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:43.394522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:43.394984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:43.395683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:43.396498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:43.397080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:43.399241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:43.419123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 272/536] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:43.419330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:43.424415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:43.487399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24719139-9de5-404a-891f-498528e21e92 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:43.529832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:43.530484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:43.530767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:43.531037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:43.531818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=6c07c85b63514d10bb17b836c60f2824) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:43.532215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:43.533754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:43.539965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 265/537] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:43 2022] GET /identity/v3/role_assignments?user.id=6c07c85b63514d10bb17b836c60f2824&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:43.540944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:43.557320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c9211-c4be-43db-9872-dc76aaca0a69 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:43.608892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:43.609956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:43.610266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:43.610538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:43.611285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=817a190d001b40a2afee944f19270aee) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:43.611593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:43.613172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:43.619731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 273/538] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:43 2022] GET /identity/v3/role_assignments?user.id=817a190d001b40a2afee944f19270aee&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:43.682718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:43.716471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b226a3-0397-443d-b436-15d66c8c2835 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:43.770721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:43.771369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:43.771645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:43.771918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:43.783596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:43.785078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:43.797646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:43.799796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:43.802014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 266/539] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:43.804776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:43.897249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:43.949217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32b335-a0ff-47c1-818c-11576c301611 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:43.995452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea17f39-fe05-4914-a7be-184c041f8da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:43.996879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea17f39-fe05-4914-a7be-184c041f8da3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:43.997368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea17f39-fe05-4914-a7be-184c041f8da3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:43.997872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea17f39-fe05-4914-a7be-184c041f8da3 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:43.999597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea17f39-fe05-4914-a7be-184c041f8da3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:44.000668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea17f39-fe05-4914-a7be-184c041f8da3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:44.003015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea17f39-fe05-4914-a7be-184c041f8da3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:44.017255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 274/540] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:44.022913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:44.181100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b2c06-20aa-41f9-9c6f-97c6a9bfaab3 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:44.215934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583d0a0-b30a-48d2-9844-6d41c990dc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:44.216542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d0a0-b30a-48d2-9844-6d41c990dc0e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:44.216820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d0a0-b30a-48d2-9844-6d41c990dc0e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:44.217087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d0a0-b30a-48d2-9844-6d41c990dc0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:44.217815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583d0a0-b30a-48d2-9844-6d41c990dc0e None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=a71d353c1bd849028930261e9306fa01) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:44.218119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583d0a0-b30a-48d2-9844-6d41c990dc0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:44.219610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583d0a0-b30a-48d2-9844-6d41c990dc0e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:44.226062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 267/541] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:44 2022] GET /identity/v3/role_assignments?user.id=a71d353c1bd849028930261e9306fa01&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:44.753978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c099559-0e49-4d3c-ae8f-13b55ec6b9ca None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:44.754356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c099559-0e49-4d3c-ae8f-13b55ec6b9ca None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:44.754566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c099559-0e49-4d3c-ae8f-13b55ec6b9ca None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:44.756174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 275/542] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:44.761617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbec73b-f9b0-46ce-ae20-7556b62edda0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:44.761906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbec73b-f9b0-46ce-ae20-7556b62edda0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:44.762199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbec73b-f9b0-46ce-ae20-7556b62edda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:44.769162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:44.781541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-bbbec73b-f9b0-46ce-ae20-7556b62edda0 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:44.782776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bbbec73b-f9b0-46ce-ae20-7556b62edda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:44.853110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 268/543] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:45.066221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f88a37-eb9d-489f-a09b-410cc24f3601 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:45.066705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f88a37-eb9d-489f-a09b-410cc24f3601 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:45.066705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f88a37-eb9d-489f-a09b-410cc24f3601 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:45.068257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 276/544] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:45.073496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe31c-76b1-4e3a-8c25-60382ccad8f7 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:45.073891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe31c-76b1-4e3a-8c25-60382ccad8f7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:45.074056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe31c-76b1-4e3a-8c25-60382ccad8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:45.092512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-77dfe31c-76b1-4e3a-8c25-60382ccad8f7 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:45.093745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-77dfe31c-76b1-4e3a-8c25-60382ccad8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:45.162271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 269/545] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:45.277093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f88a37-eb9d-489f-a09b-410cc24f3601 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:45.397126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f88a37-eb9d-489f-a09b-410cc24f3601 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:45.424881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:45.434214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0434e-cd9a-49f0-89f0-e305e7f16153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:45.435038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0434e-cd9a-49f0-89f0-e305e7f16153 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:45.435598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0434e-cd9a-49f0-89f0-e305e7f16153 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:45.436030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0434e-cd9a-49f0-89f0-e305e7f16153 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:45.450034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0434e-cd9a-49f0-89f0-e305e7f16153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-01c0434e-cd9a-49f0-89f0-e305e7f16153 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:45.455349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:45.456256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:45.457222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 277/546] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:45.463971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe31c-76b1-4e3a-8c25-60382ccad8f7 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:45.531613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:45.553222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0434e-cd9a-49f0-89f0-e305e7f16153 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:45.574117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe31c-76b1-4e3a-8c25-60382ccad8f7 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:45.613435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:45.614047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:45.614354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:45.614622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:45.616034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:45.616330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:45.617692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:45.634405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 270/547] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:45.642267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:45.769298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0434e-cd9a-49f0-89f0-e305e7f16153 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:45.775974 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a84bd-e289-48dd-ad52-333d4f2e9391 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:45.799891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:45.810685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:45.811398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:45.813065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:45.813065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:45.823750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:45.825397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:45.840621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:45.841661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:45.843056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:45.843056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:45.843056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:45.843056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 271/548] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:45.848145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:45.856184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c727-a830-43c5-96dc-bb6414e27be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:45.856825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c727-a830-43c5-96dc-bb6414e27be3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:45.857110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c727-a830-43c5-96dc-bb6414e27be3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:45.857391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c727-a830-43c5-96dc-bb6414e27be3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:45.887342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0c727-a830-43c5-96dc-bb6414e27be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-33b0c727-a830-43c5-96dc-bb6414e27be3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:45.893016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:45.893906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:45.894763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:45.900617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 278/549] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:33:45.901717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c727-a830-43c5-96dc-bb6414e27be3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:45.954447 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d8ab7-13f8-41f9-b258-2b23a0394111 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:45.991730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:45.992087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:45.992384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:45.992669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:45.993003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:45.993654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:45.993997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:45.995480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:46.008871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 272/550] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:46.014752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:46.094524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:46.115925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c727-a830-43c5-96dc-bb6414e27be3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:46.175317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eb0a4-bf66-4e6c-861f-73f7d8b365f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:46.176369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb0a4-bf66-4e6c-861f-73f7d8b365f6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.176651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb0a4-bf66-4e6c-861f-73f7d8b365f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.176917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb0a4-bf66-4e6c-861f-73f7d8b365f6 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.177846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eb0a4-bf66-4e6c-861f-73f7d8b365f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:46.178189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eb0a4-bf66-4e6c-861f-73f7d8b365f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:46.179894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eb0a4-bf66-4e6c-861f-73f7d8b365f6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:46.196547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 279/551] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:46.196547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6a2d-46cd-43dc-9b79-844754f4bc15 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.196547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6a2d-46cd-43dc-9b79-844754f4bc15 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.196547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6a2d-46cd-43dc-9b79-844754f4bc15 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.196547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 280/552] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:46.197467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6a2d-46cd-43dc-9b79-844754f4bc15 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:46.201896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec309b-50e0-40bf-832b-c45803bc00c2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:46.243693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26424f-2e91-48dc-aaa0-46e8e682e9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:46.244655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26424f-2e91-48dc-aaa0-46e8e682e9d0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.245134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26424f-2e91-48dc-aaa0-46e8e682e9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.246026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26424f-2e91-48dc-aaa0-46e8e682e9d0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.254970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26424f-2e91-48dc-aaa0-46e8e682e9d0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:46.257013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26424f-2e91-48dc-aaa0-46e8e682e9d0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-cf26424f-2e91-48dc-aaa0-46e8e682e9d0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:46.271397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:46.272735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:33:46.273840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:46.278699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 273/553] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:33:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:33:46.278699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43286e-fa1e-4c9b-83d7-7d30c49303c0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.278699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43286e-fa1e-4c9b-83d7-7d30c49303c0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.278699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43286e-fa1e-4c9b-83d7-7d30c49303c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.296176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9a43286e-fa1e-4c9b-83d7-7d30c49303c0 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:46.297528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9a43286e-fa1e-4c9b-83d7-7d30c49303c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:46.324091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6a2d-46cd-43dc-9b79-844754f4bc15 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:46.336027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:46.365141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:46.366241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.366725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.367200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.378361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 274/554] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:46.379394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43286e-fa1e-4c9b-83d7-7d30c49303c0 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:46.382972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:46.384651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:46.401220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:46.402234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:46.403196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:46.403196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:46.403196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:46.405373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 281/555] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:46.411310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:46.415569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:46.537693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2bb45-afe1-4ac1-9125-45afcef01126 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:46.550662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43286e-fa1e-4c9b-83d7-7d30c49303c0 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:46.574708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71efaab-e949-475c-8ce6-6c49a4fc2c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:46.575726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efaab-e949-475c-8ce6-6c49a4fc2c3b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.576473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efaab-e949-475c-8ce6-6c49a4fc2c3b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.577063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efaab-e949-475c-8ce6-6c49a4fc2c3b None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.578473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71efaab-e949-475c-8ce6-6c49a4fc2c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:46.578973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71efaab-e949-475c-8ce6-6c49a4fc2c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:46.580722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71efaab-e949-475c-8ce6-6c49a4fc2c3b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:46.585715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ee8b-5532-4fc1-a606-7c5c77dd2935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:46.586354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ee8b-5532-4fc1-a606-7c5c77dd2935 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.586637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ee8b-5532-4fc1-a606-7c5c77dd2935 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.586912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ee8b-5532-4fc1-a606-7c5c77dd2935 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.587638 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1ee8b-5532-4fc1-a606-7c5c77dd2935 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:46.587943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1ee8b-5532-4fc1-a606-7c5c77dd2935 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:46.589417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1ee8b-5532-4fc1-a606-7c5c77dd2935 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:46.597009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 282/556] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:46.598804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb0bf4-443a-4041-876c-e7f0bae3d443 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.599286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb0bf4-443a-4041-876c-e7f0bae3d443 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.599801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb0bf4-443a-4041-876c-e7f0bae3d443 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.605447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 283/557] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:46.606634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb0bf4-443a-4041-876c-e7f0bae3d443 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:46.609063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 275/558] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 13:33:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:46.610388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4304-0efe-41da-b42c-d7bb8fc1bcc9 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.610673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4304-0efe-41da-b42c-d7bb8fc1bcc9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.610962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4304-0efe-41da-b42c-d7bb8fc1bcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.636999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:46.639076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3b8f4304-0efe-41da-b42c-d7bb8fc1bcc9 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:46.641585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3b8f4304-0efe-41da-b42c-d7bb8fc1bcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:46.742382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 276/559] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:46.742382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f4304-0efe-41da-b42c-d7bb8fc1bcc9 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:46.755675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb0bf4-443a-4041-876c-e7f0bae3d443 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:46.761796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:46.793816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:46.794625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.794969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.795309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.802966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:46.804523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:46.816582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:46.817567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:33:46.818568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:46.822561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 284/560] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:33:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:46.823445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:46.867828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f4304-0efe-41da-b42c-d7bb8fc1bcc9 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:46.913415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51aacb-a12a-4237-845a-63910be7d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:46.914101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51aacb-a12a-4237-845a-63910be7d656 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.914487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51aacb-a12a-4237-845a-63910be7d656 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.914891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51aacb-a12a-4237-845a-63910be7d656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:46.922397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:46.943427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bddd5-7f79-4e30-bfb5-fc7bbcad7c0f None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:46.983305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8ed1e-dc0f-4763-8df5-5eaaa802aee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:46.984967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ed1e-dc0f-4763-8df5-5eaaa802aee0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:46.985481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ed1e-dc0f-4763-8df5-5eaaa802aee0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:46.985981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ed1e-dc0f-4763-8df5-5eaaa802aee0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.001113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8ed1e-dc0f-4763-8df5-5eaaa802aee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-27f8ed1e-dc0f-4763-8df5-5eaaa802aee0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:47.006467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:47.007404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:47.008251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 285/561] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:47.009562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8ed1e-dc0f-4763-8df5-5eaaa802aee0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.026473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51aacb-a12a-4237-845a-63910be7d656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.028061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51aacb-a12a-4237-845a-63910be7d656 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.047486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:47.123895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8ed1e-dc0f-4763-8df5-5eaaa802aee0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.139420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 277/562] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:47.140814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5c474-0eab-4839-95de-3b7f6d803a98 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.141105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5c474-0eab-4839-95de-3b7f6d803a98 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.141380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5c474-0eab-4839-95de-3b7f6d803a98 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.142648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 278/563] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:47.143336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5c474-0eab-4839-95de-3b7f6d803a98 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.162130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.162764 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.163044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.163316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.164097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:47.164354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.165795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.175622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:47.183959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 286/564] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:47.184856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.245410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5c474-0eab-4839-95de-3b7f6d803a98 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.259224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:47.278530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.279137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.279418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.279694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.280353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:47.281214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.282653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.284915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f14366-473a-499d-9453-0ffbe95f1635 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.290617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 279/565] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:47.291419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.318858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20289a34-2c24-47de-b7d8-83a79cf94aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.319487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289a34-2c24-47de-b7d8-83a79cf94aee None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.319774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289a34-2c24-47de-b7d8-83a79cf94aee None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.320047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289a34-2c24-47de-b7d8-83a79cf94aee None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.330588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20289a34-2c24-47de-b7d8-83a79cf94aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-20289a34-2c24-47de-b7d8-83a79cf94aee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:47.335369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:47.336349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:47.337218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 287/566] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:47.337917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f21bb5-efa8-4270-ac3f-71f12052840b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.338187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f21bb5-efa8-4270-ac3f-71f12052840b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.338493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f21bb5-efa8-4270-ac3f-71f12052840b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.356485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-50f21bb5-efa8-4270-ac3f-71f12052840b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:47.357698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-50f21bb5-efa8-4270-ac3f-71f12052840b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:47.399156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9f0fa-4423-4a29-adba-e1a6d7b35bac None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.411420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:47.434016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.434770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.434988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.435251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 288/567] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:47.435469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.436100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f21bb5-efa8-4270-ac3f-71f12052840b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.448001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.450546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:47.469642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:47.471011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:47.476921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:47.476921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:47.476921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:47.476921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 280/568] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:47.476921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.507264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:47.559371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f21bb5-efa8-4270-ac3f-71f12052840b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.581489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31267513-3941-4a72-a963-964d1f55e28b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.594615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.595345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.595692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.596054 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.607023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.609230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.621364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.622048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.622431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.622764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:47.623189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:47.624841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:47.625975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:47.625975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:47.625975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:47.625975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 289/569] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:47.625975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.638308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.640094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:47.656214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:47.657220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:47.658122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:47.658122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:47.658122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:47.661961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 281/570] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:47.662796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.677396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:47.738409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e6f00-2e23-431a-856c-9be9c5567181 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.745774 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:47.764353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aecfa-2170-4d70-8579-0ef410ed15be None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.771753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.772436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.772717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.773013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.773672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:47.773974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.775396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.784026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 290/571] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:47.784727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.806274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.806880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.807163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.807437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.808085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:47.808385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.809748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.829820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 282/572] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:47.830954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:47.927009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8901709-65b3-41a8-a923-f726a9d2e003 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.928551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:47.959257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a6c95-83b5-488f-a990-d9a21f31cb8d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:47.965114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:47.965716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:47.965998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:47.966301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:47.966942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:47.967246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:47.968631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:47.981614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 291/573] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:47.982723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.002537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.003078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.003452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.003750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.004397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:48.004695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.008094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.016729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:48.020061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 283/574] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:48.020942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.095085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c519e3-072f-40cc-bddc-e1c6b790f0a4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.131520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb74049-b617-4d0f-870c-a5a5df5986d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.132143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb74049-b617-4d0f-870c-a5a5df5986d7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.132417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb74049-b617-4d0f-870c-a5a5df5986d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.132690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb74049-b617-4d0f-870c-a5a5df5986d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.132912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361fec6-bef9-4fd9-91de-484c96606fb4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.145325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb74049-b617-4d0f-870c-a5a5df5986d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-8eb74049-b617-4d0f-870c-a5a5df5986d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:48.150387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:48.151362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:48.152198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 292/575] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:48.153291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb74049-b617-4d0f-870c-a5a5df5986d7 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.173871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.174537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.174820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.175094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.179273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:48.185833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.187878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:48.202602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:48.203660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:48.204586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:48.204586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:48.204586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:48.204586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 284/576] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:48.204909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.261269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:48.263680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb74049-b617-4d0f-870c-a5a5df5986d7 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.297760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.298489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.298777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.299052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.305554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.307030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:48.318635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:48.319665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 293/577] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:48.320707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.382545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c4122-274d-4320-a487-2e8060e4a3a1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.400026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:48.415812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.416443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.416724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.416996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.417519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d94afc-97d5-4347-bdb0-955eb20f9a81 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.424094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.425533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:48.437320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:48.438262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 285/578] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:48.439440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.453638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.454303 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.454586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.454858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.461879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.463355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:48.475719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:48.476804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 294/579] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:48.478305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.545346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831598d-f2eb-4ccc-aa7d-766e3011723c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.558794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:48.575858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6e622-abdb-4974-bf0a-1957a5b56cb2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.579224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.579828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.580184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.580470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.581471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:48.581833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.583657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.592498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 286/580] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:48.592498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.610457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.611066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.611346 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.611616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.612254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:48.612562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.613926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.625995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 295/581] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:48.626814 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.629248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:48.700508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2fbcf-261c-43ab-9f1a-97cb01b3dfe3 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.726807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d995b3-0fbd-4e7d-8bc7-1f09c4adcf1c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.736068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.736674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.736955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.737227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.737922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:48.738315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.739766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.752123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 287/582] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:48.752928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.765806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.766442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.766725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.767000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.767695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:48.768003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.769426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.781596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 296/583] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:48.782429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.786922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:48.842561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:48.852335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33b277-83ab-4fb0-8c05-c03b9dc60b0b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.880949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76517b-52fe-452d-962d-f1849fb7fbee None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:48.885568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.886197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.886485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.886758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.887491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:48.887849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.889272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.907312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 288/584] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:48.908120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:48.918660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:48.919294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:48.919578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:48.919849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:48.930122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:48.931597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:48.945253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:48.946410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:48.947350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:48.947350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:48.947350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:48.947350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 297/585] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:48.947576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.001193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:49.011260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b3b19-c40b-4057-95de-c745a48334a3 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.048048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.048668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.048964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.049224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.052212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ff149-7d1f-41ef-8f29-cadb1ce64d76 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.056440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.057884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.066341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:49.070904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:49.074426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 289/586] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:49.075427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.090895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.091506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.091812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.092088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.093541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=136ce464d3194f98952beed2d89b8bc3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:49.093785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.095574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.100933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 298/587] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:48 2022] GET /identity/v3/role_assignments?user.id=136ce464d3194f98952beed2d89b8bc3&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:49.101701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.188918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73b057-b47b-4083-b989-525a0287efcc None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.214473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31c010-44e7-4f9a-b9db-1628e4d2df3c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.227817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09108ad-33c0-48c0-8815-85cdcdf645cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.228517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09108ad-33c0-48c0-8815-85cdcdf645cf None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.228783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09108ad-33c0-48c0-8815-85cdcdf645cf None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.229055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09108ad-33c0-48c0-8815-85cdcdf645cf None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/c24d70e571c64f018aa199c4c08d153e/roles/04bab2a5f02f4140b88885a57b278b5d` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.247830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09108ad-33c0-48c0-8815-85cdcdf645cf None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=c24d70e571c64f018aa199c4c08d153e, role_id=04bab2a5f02f4140b88885a57b278b5d)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.250762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09108ad-33c0-48c0-8815-85cdcdf645cf None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.255515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.256222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.256504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.256774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.268372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.269859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:49.284873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:49.286045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:49.287088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:49.287088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:49.287088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:49.287088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 299/588] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:49.287088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.318649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:49.327129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 290/589] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:33:49 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/c24d70e571c64f018aa199c4c08d153e/roles/04bab2a5f02f4140b88885a57b278b5d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:49.328029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09108ad-33c0-48c0-8815-85cdcdf645cf None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.337736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:49.425339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d6674-37fe-4fb7-a566-7d46cf4b42ca None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.442635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09108ad-33c0-48c0-8815-85cdcdf645cf None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.465990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.467068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.467502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.468483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.469555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:49.470014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.471825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.483352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.484495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.484773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.485039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.485844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:49.486129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.487670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.488532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 300/590] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:49.489758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.513400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 291/591] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:49.513400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.573208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:49.606123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213e0e4-82ef-42d2-837b-e305bb4eda28 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.618380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:49.619130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412cbe7-9106-4607-b6a1-de7b1647af40 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.646058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.647025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.647499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.647962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.648794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:49.649290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.650890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.656930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.657569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.657851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.658126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.664855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 301/592] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:49.669907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.669907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.670595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:49.683857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:49.685064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:49.686083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 292/593] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:49.687349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.777671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9f03c-2f33-4349-9b93-e4e944516b9d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.788171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc5012-4bde-4825-baf0-1d7cce55d3ab None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:49.814453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.815488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.815989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.816474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.823425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:49.824053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:49.824329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:49.824598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:49.830152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.831922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.832544 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:49.833987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:49.847024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:49.847999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:49.849158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:49.849158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:49.849158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:49.849158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:49.849158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:49.849158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:49.849158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:49.849158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:49.851944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:49.852891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:49.853777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:49.853777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:49.853777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:49.853960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:49.853960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 293/594] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:49.853960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.856017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 302/595] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:49.860802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:49.955475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:49.968552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984871-2104-4fcd-93c2-1888f5d80de4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.003725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2975b8-0e2e-4a6e-92db-6258e61a93f1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.006121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.006932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.007216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.007486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.008752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:50.008752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.010083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.034288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 294/596] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:50.035207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.052490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.053722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.054254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.054987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.056311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:50.056829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.058513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.078763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 303/597] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:50.079368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.085413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:50.157869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043060-47eb-4478-99d2-89e68b19062a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.190586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:50.195427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.196114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.196457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.196828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.197576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:50.197950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.199450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.203256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ab851-dcef-40a5-88ef-aa8e39a6382c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.214340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 295/598] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:50.215254 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.251264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.252663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.253355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.254607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.273087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.276095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:50.301110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:50.302181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:50.303082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:50.303082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:50.303082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:50.306217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 304/599] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:50.307314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.331554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:50.336373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d40bc3-c049-4997-ae18-e9320f664655 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.377699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c35749-fbe3-4c61-be1d-95b493c2f025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.379067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c35749-fbe3-4c61-be1d-95b493c2f025 None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.379977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c35749-fbe3-4c61-be1d-95b493c2f025 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.380745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c35749-fbe3-4c61-be1d-95b493c2f025 None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/6c07c85b63514d10bb17b836c60f2824/roles/04bab2a5f02f4140b88885a57b278b5d` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.399882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c35749-fbe3-4c61-be1d-95b493c2f025 None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=6c07c85b63514d10bb17b836c60f2824, role_id=04bab2a5f02f4140b88885a57b278b5d)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.402275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c35749-fbe3-4c61-be1d-95b493c2f025 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.425908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ecddd-fab9-454b-ae4e-b8dd3da6c59e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.440327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:50.460003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.460620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.460898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.461170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.468730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.470418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.474180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 296/600] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:33:50 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/6c07c85b63514d10bb17b836c60f2824/roles/04bab2a5f02f4140b88885a57b278b5d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:50.476984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c35749-fbe3-4c61-be1d-95b493c2f025 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:50.495322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:50.496348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:50.497262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:50.497262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:50.497262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:50.497262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:50.497262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:50.497262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:50.497262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:50.497262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:50.497694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 305/601] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:50.502564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.609326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c35749-fbe3-4c61-be1d-95b493c2f025 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.659000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.660010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.660498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.660963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.661814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:50.662404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.664100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.680173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 297/602] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:50.684677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.702839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:50.747754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62185a-34f1-40f5-931a-fef8cd79c920 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.797907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.798574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.798860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.799490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.800210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:50.800518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.801948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.818443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c94657-3713-4758-890a-6a673d482085 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:50.818759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 306/603] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:50.824542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.862717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:50.877769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:50.878616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:50.879002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:50.879371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:50.887101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:50.889120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:50.910511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:50.911539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:50.912442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 298/604] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:33:50.919532 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:50.960122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36dcb3-b096-4634-bbf4-b207a73d91bf None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:51.027431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1466d-29e7-4410-9128-6561171b923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:51.028654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1466d-29e7-4410-9128-6561171b923a None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:51.029034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1466d-29e7-4410-9128-6561171b923a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:51.029538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1466d-29e7-4410-9128-6561171b923a None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/817a190d001b40a2afee944f19270aee/roles/04bab2a5f02f4140b88885a57b278b5d` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:51.038711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:51.054453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1466d-29e7-4410-9128-6561171b923a None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=817a190d001b40a2afee944f19270aee, role_id=04bab2a5f02f4140b88885a57b278b5d)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:51.055921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1466d-29e7-4410-9128-6561171b923a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:51.082115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66b0ec-0244-4326-b6bc-387e739f5b61 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:51.120363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:51.121338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:51.122016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:51.122577 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:51.123514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:51.124256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:51.125990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:51.140237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 299/605] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:51.149096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:51.169593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 307/606] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:33:50 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/817a190d001b40a2afee944f19270aee/roles/04bab2a5f02f4140b88885a57b278b5d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:51.272609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0fb84-a50f-4ac7-ba31-ef805ccb7fb2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:51.303790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:51.316984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888263ef-435c-404b-b17a-2128797cad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:51.317664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888263ef-435c-404b-b17a-2128797cad1e None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:51.317987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888263ef-435c-404b-b17a-2128797cad1e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:51.318287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888263ef-435c-404b-b17a-2128797cad1e None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/a71d353c1bd849028930261e9306fa01/roles/04bab2a5f02f4140b88885a57b278b5d` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:51.335285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888263ef-435c-404b-b17a-2128797cad1e None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=a71d353c1bd849028930261e9306fa01, role_id=04bab2a5f02f4140b88885a57b278b5d)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:51.336789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888263ef-435c-404b-b17a-2128797cad1e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:51.361357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18212b-bac0-4cf7-8818-feb109de2439 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:51.362397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18212b-bac0-4cf7-8818-feb109de2439 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:51.362882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18212b-bac0-4cf7-8818-feb109de2439 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:51.364907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 308/607] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:33:51.370904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72649407-8124-4fb5-9028-b37e17dfc80d None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:51.371516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72649407-8124-4fb5-9028-b37e17dfc80d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:51.371992 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72649407-8124-4fb5-9028-b37e17dfc80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:51.393278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-72649407-8124-4fb5-9028-b37e17dfc80d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:51.394632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-72649407-8124-4fb5-9028-b37e17dfc80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:51.430656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 300/608] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:33:51 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/a71d353c1bd849028930261e9306fa01/roles/04bab2a5f02f4140b88885a57b278b5d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:51.507617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 309/609] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:51.821750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888263ef-435c-404b-b17a-2128797cad1e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:51.877003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57a293-f01d-42c0-870c-c62e74290160 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:51.877336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57a293-f01d-42c0-870c-c62e74290160 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:51.877620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57a293-f01d-42c0-870c-c62e74290160 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:51.879170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 310/610] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:51.884374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a14dc-a295-47e5-9210-c03943740e01 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:51.884664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a14dc-a295-47e5-9210-c03943740e01 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:51.884943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a14dc-a295-47e5-9210-c03943740e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:51.904449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-be6a14dc-a295-47e5-9210-c03943740e01 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:51.906159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-be6a14dc-a295-47e5-9210-c03943740e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:51.916424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:51.962775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888263ef-435c-404b-b17a-2128797cad1e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:52.020884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 311/611] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:33:52.028589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df5645-9571-4ed4-9388-e7dd32a0d26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:52.029235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df5645-9571-4ed4-9388-e7dd32a0d26e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:52.029523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df5645-9571-4ed4-9388-e7dd32a0d26e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:52.030366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df5645-9571-4ed4-9388-e7dd32a0d26e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:52.047590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df5645-9571-4ed4-9388-e7dd32a0d26e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-52df5645-9571-4ed4-9388-e7dd32a0d26e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:52.052481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:52.053421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:52.054339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 301/612] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:52.059613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a14dc-a295-47e5-9210-c03943740e01 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:52.221280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a14dc-a295-47e5-9210-c03943740e01 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:52.268963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:52.269572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:52.269852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:52.270126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:52.270819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:52.271127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:52.272493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:52.285840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 312/613] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:33:52.291418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df5645-9571-4ed4-9388-e7dd32a0d26e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:52.348659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:52.373180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:52.383940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:52.474518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df5645-9571-4ed4-9388-e7dd32a0d26e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:52.533333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a8a5f-ed10-4d8c-a6ad-1908428e94e4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:52.544716 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:52.549357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:52.549935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:52.550547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:52.574504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:52.577053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:52.591202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284deee4-5fd9-4428-a96b-8b28bddf5976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:52.591995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284deee4-5fd9-4428-a96b-8b28bddf5976 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:52.595140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284deee4-5fd9-4428-a96b-8b28bddf5976 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:52.595140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284deee4-5fd9-4428-a96b-8b28bddf5976 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:52.604636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284deee4-5fd9-4428-a96b-8b28bddf5976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-284deee4-5fd9-4428-a96b-8b28bddf5976 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:52.610193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:52.611161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:52.612006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:52.612643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:52.613476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:52.615453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:52.615453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:52.615453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:52.615453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 302/614] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:52.619136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:52.626247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 313/615] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:52.634892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284deee4-5fd9-4428-a96b-8b28bddf5976 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:52.757204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:52.798519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:52.814041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bee599-004b-4f21-9e73-a67fb618f92e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:52.834612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284deee4-5fd9-4428-a96b-8b28bddf5976 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:52.874460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:52.875126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:52.875414 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:52.875688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:52.876366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:52.876669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:52.878119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:52.891222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a5760-995e-4478-b766-e91311a773ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:52.891843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a5760-995e-4478-b766-e91311a773ea None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:52.892126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a5760-995e-4478-b766-e91311a773ea None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:52.892404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a5760-995e-4478-b766-e91311a773ea None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:52.893059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a5760-995e-4478-b766-e91311a773ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:52.893366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a5760-995e-4478-b766-e91311a773ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:52.893482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 314/616] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:52.894729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:52.894839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a5760-995e-4478-b766-e91311a773ea None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:52.945936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 303/617] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:52.948713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a5760-995e-4478-b766-e91311a773ea None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:53.104141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c5813-77d4-4e8f-97cf-f8f4da648fae None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:53.133945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a5760-995e-4478-b766-e91311a773ea None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:53.137841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:53.156884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:53.159705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.159705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.159705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.178023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:53.180788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:53.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:53.203467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:53.204371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:53.204371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:53.204371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:53.209114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b14f87-f5d7-4ef8-8747-e3fce57f8a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:53.209984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b14f87-f5d7-4ef8-8747-e3fce57f8a1f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.210504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b14f87-f5d7-4ef8-8747-e3fce57f8a1f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.211073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b14f87-f5d7-4ef8-8747-e3fce57f8a1f None admin] PATH_INFO: `/v3/users/swift` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.212748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 315/618] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:53.220842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:53.222785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b14f87-f5d7-4ef8-8747-e3fce57f8a1f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:53.226307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b14f87-f5d7-4ef8-8747-e3fce57f8a1f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-d3b14f87-f5d7-4ef8-8747-e3fce57f8a1f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:53.248071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:53.249111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:33:53.250076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:53.253126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 304/619] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:33:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:53.254298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbc04a-3530-470c-8b31-ed50dea9d0db None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.254583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbc04a-3530-470c-8b31-ed50dea9d0db None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.254859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbc04a-3530-470c-8b31-ed50dea9d0db None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.256226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 305/620] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:53.260614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbc04a-3530-470c-8b31-ed50dea9d0db None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:53.273354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:53.358579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821931a-ea90-4910-bf16-31a99901665f None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:53.383505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbc04a-3530-470c-8b31-ed50dea9d0db None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:53.394096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a7d53-da66-445a-a953-a5d15e34010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:53.394845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a7d53-da66-445a-a953-a5d15e34010a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.395194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a7d53-da66-445a-a953-a5d15e34010a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.395541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a7d53-da66-445a-a953-a5d15e34010a None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.396256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a7d53-da66-445a-a953-a5d15e34010a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:53.396628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a7d53-da66-445a-a953-a5d15e34010a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:53.398064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a7d53-da66-445a-a953-a5d15e34010a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:53.415182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 316/621] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:53.416444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f9ec9-3acd-4a62-8198-0ddf354e88c1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.416806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f9ec9-3acd-4a62-8198-0ddf354e88c1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.417143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f9ec9-3acd-4a62-8198-0ddf354e88c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.428422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:53.429499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.430034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.430553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.432053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:53.432620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:53.434574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:53.437840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-617f9ec9-3acd-4a62-8198-0ddf354e88c1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:53.439846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-617f9ec9-3acd-4a62-8198-0ddf354e88c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:53.458756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 306/622] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 13:33:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:53.459937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:53.487541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:53.532099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 317/623] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:53.533041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f9ec9-3acd-4a62-8198-0ddf354e88c1 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:53.548529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:53.626200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb64309-4c47-4852-b7c8-7575c75d9ca4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:53.704473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f9ec9-3acd-4a62-8198-0ddf354e88c1 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:53.722834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:53.723746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.724288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.724737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.739198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:53.740848 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:53.742356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:53.743164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.743518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.743861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.758173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:53.758173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:53.764593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:53.765776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:33:53.766984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:53.769158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 307/624] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:33:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:53.774355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:53.776984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:53.777995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:53.778953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:53.778953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:53.778953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:53.778953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 318/625] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:53.789000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:53.838957 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:53.918293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e5eeb-0725-4d93-97d5-4f1ca79c9417 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:53.931981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec54e0d-21d6-4789-b073-34488026d7d5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:53.948386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:53.957703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:53.962235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.962235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.962235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.962235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:53.962235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:53.962235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:53.972981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ee3d0-f2bd-4096-9d8a-79bf5cf01f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:53.973974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee3d0-f2bd-4096-9d8a-79bf5cf01f6e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.974308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee3d0-f2bd-4096-9d8a-79bf5cf01f6e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.974589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee3d0-f2bd-4096-9d8a-79bf5cf01f6e None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.975274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ee3d0-f2bd-4096-9d8a-79bf5cf01f6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:53.975586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ee3d0-f2bd-4096-9d8a-79bf5cf01f6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:53.977025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ee3d0-f2bd-4096-9d8a-79bf5cf01f6e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:53.986818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 308/626] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:33:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:53.987178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:53.993110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 319/627] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:53.994670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7a0b-a156-47ac-bd3f-5592670f1c38 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:53.995157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7a0b-a156-47ac-bd3f-5592670f1c38 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:53.996008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b7a0b-a156-47ac-bd3f-5592670f1c38 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:53.998112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 320/628] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:53.999574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b7a0b-a156-47ac-bd3f-5592670f1c38 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.088060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:54.103856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5707-0130-4f99-9e8d-ef44b14a62fb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.107122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b7a0b-a156-47ac-bd3f-5592670f1c38 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.139083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0dace-2dbe-43fb-998f-594ce89a2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.139710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0dace-2dbe-43fb-998f-594ce89a2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.139990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0dace-2dbe-43fb-998f-594ce89a2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.140265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0dace-2dbe-43fb-998f-594ce89a2fb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.142910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2f585-c80a-40d9-bf69-719bd02407ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.143511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f585-c80a-40d9-bf69-719bd02407ee None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.143798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f585-c80a-40d9-bf69-719bd02407ee None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.144069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f585-c80a-40d9-bf69-719bd02407ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.150052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0dace-2dbe-43fb-998f-594ce89a2fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-73d0dace-2dbe-43fb-998f-594ce89a2fb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:54.154900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:54.155935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 309/629] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:54.156803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0dace-2dbe-43fb-998f-594ce89a2fb4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.207020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:54.213333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2f585-c80a-40d9-bf69-719bd02407ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:54.216770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2f585-c80a-40d9-bf69-719bd02407ee None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:54.259352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0dace-2dbe-43fb-998f-594ce89a2fb4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.294048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8eb9c-8956-417c-ae07-d548b3691dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.294708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8eb9c-8956-417c-ae07-d548b3691dc0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.294988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8eb9c-8956-417c-ae07-d548b3691dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.295257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8eb9c-8956-417c-ae07-d548b3691dc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.295805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:54.301871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8eb9c-8956-417c-ae07-d548b3691dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:54.303342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8eb9c-8956-417c-ae07-d548b3691dc0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:54.314001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 321/630] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f9c8eb9c-8956-417c-ae07-d548b3691dc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:54.314475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:54.315399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:54.316327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:54.316327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:54.316327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:54.316327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:54.316327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:54.316327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:54.316327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:54.316327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:54.316779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ea684-059d-4581-9de8-93c2c1f18793 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.316779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ea684-059d-4581-9de8-93c2c1f18793 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.316779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ea684-059d-4581-9de8-93c2c1f18793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.316984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 310/631] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:54.317573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffeb86-5407-4b1d-a334-12c31aae6caf None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.317573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffeb86-5407-4b1d-a334-12c31aae6caf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.317745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffeb86-5407-4b1d-a334-12c31aae6caf None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.319204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 311/632] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:54.324005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffeb86-5407-4b1d-a334-12c31aae6caf None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.334842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-490ea684-059d-4581-9de8-93c2c1f18793 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:54.336046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-490ea684-059d-4581-9de8-93c2c1f18793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:54.403024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 322/633] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:54.403857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ea684-059d-4581-9de8-93c2c1f18793 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.429976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffeb86-5407-4b1d-a334-12c31aae6caf None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.439124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:54.465755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.466339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.466705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.467510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.468288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:54.468852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:54.470347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:54.484693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 312/634] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:54.485568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.519874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ea684-059d-4581-9de8-93c2c1f18793 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.533823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:54.555155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c37e7-439c-4f46-8f47-dfc74f8a6527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.555855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c37e7-439c-4f46-8f47-dfc74f8a6527 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.556132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c37e7-439c-4f46-8f47-dfc74f8a6527 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.556448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c37e7-439c-4f46-8f47-dfc74f8a6527 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.557512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c37e7-439c-4f46-8f47-dfc74f8a6527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:54.558081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c37e7-439c-4f46-8f47-dfc74f8a6527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:54.560220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c37e7-439c-4f46-8f47-dfc74f8a6527 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:54.572683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 323/635] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:54.574856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e08a8-a04b-4394-bd5c-2876cbf745d1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.575357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e08a8-a04b-4394-bd5c-2876cbf745d1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.576056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e08a8-a04b-4394-bd5c-2876cbf745d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.597428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9519f-4ed0-43a9-8708-43280814b1ec None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.598604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ff8e08a8-a04b-4394-bd5c-2876cbf745d1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:54.599977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ff8e08a8-a04b-4394-bd5c-2876cbf745d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:54.632930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.633665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.636946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.637089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.648010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:54.649464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:54.664488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:54.665541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:54.666707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:54.666707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:54.666707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:54.666707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 313/636] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:54.666707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.675918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 324/637] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:54.676751 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e08a8-a04b-4394-bd5c-2876cbf745d1 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.694078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:54.777261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538674-331f-46ef-8281-170813b3e6e8 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.778538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e08a8-a04b-4394-bd5c-2876cbf745d1 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.781103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:54.814300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.814725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df534ca-7ae7-4e15-adaf-3e124b031315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.814933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.815214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.815486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.816018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df534ca-7ae7-4e15-adaf-3e124b031315 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.816482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df534ca-7ae7-4e15-adaf-3e124b031315 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:54.816909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df534ca-7ae7-4e15-adaf-3e124b031315 None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/136ce464d3194f98952beed2d89b8bc3/roles/04bab2a5f02f4140b88885a57b278b5d` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:54.827682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:54.829704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:54.831755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df534ca-7ae7-4e15-adaf-3e124b031315 None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=136ce464d3194f98952beed2d89b8bc3, role_id=04bab2a5f02f4140b88885a57b278b5d)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:54.833326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df534ca-7ae7-4e15-adaf-3e124b031315 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:54.844469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:54.850556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:54.851524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:54.851524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:54.851524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:54.851524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 314/638] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:54.851524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.906078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 325/639] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:33:54 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/136ce464d3194f98952beed2d89b8bc3/roles/04bab2a5f02f4140b88885a57b278b5d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:33:54.919135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df534ca-7ae7-4e15-adaf-3e124b031315 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:54.919341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:54.958850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd957d-f957-4518-b02e-10caea224f3d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:54.998840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:54.999473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:54.999756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.000029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.000696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:55.001002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.002435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.019680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 315/640] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:55.019680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.049490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df534ca-7ae7-4e15-adaf-3e124b031315 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.091325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4dd17-7b04-4d59-9ba5-6996fbdf8e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.092597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4dd17-7b04-4d59-9ba5-6996fbdf8e45 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.092880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4dd17-7b04-4d59-9ba5-6996fbdf8e45 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.093152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4dd17-7b04-4d59-9ba5-6996fbdf8e45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.108576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4dd17-7b04-4d59-9ba5-6996fbdf8e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-bda4dd17-7b04-4d59-9ba5-6996fbdf8e45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:55.113455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:55.114553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 326/641] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:55.115412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4dd17-7b04-4d59-9ba5-6996fbdf8e45 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.134036 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:55.155785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fe736-0d27-4810-bf4c-70072b77b8e9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.190732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:55.195556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.196220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.196505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.196776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.197423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:55.197735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.199134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.211912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 316/642] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:55.212800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.225540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4dd17-7b04-4d59-9ba5-6996fbdf8e45 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.261891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfae2e-19a4-430c-a50f-ba6e0cb74e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.262573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfae2e-19a4-430c-a50f-ba6e0cb74e36 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.262904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfae2e-19a4-430c-a50f-ba6e0cb74e36 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.263182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfae2e-19a4-430c-a50f-ba6e0cb74e36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.274091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfae2e-19a4-430c-a50f-ba6e0cb74e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-fcbfae2e-19a4-430c-a50f-ba6e0cb74e36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:55.278963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:55.280607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:55.281484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 327/643] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:55.282914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfae2e-19a4-430c-a50f-ba6e0cb74e36 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.326773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d12ae-949d-415f-8453-96ccabb8e1a9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.353416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:55.366285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.366927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.368206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.368206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.379232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.380745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.384606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfae2e-19a4-430c-a50f-ba6e0cb74e36 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:55.393888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:55.395015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 317/644] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:55.396121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.424719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.425323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.425602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.425878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.426581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:55.426884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.428256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.436453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 328/645] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:55.437273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.462230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:55.519412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2e76-345a-429a-aea2-592e08a11118 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.542239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400fc3e-a179-4b21-a916-3d515561b4e9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.556262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.557328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.557633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.558060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.567016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.568613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.578321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.578893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.579176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.579448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.580113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:55.580414 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:55.580691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:55.581593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:33:55.582921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:55.583478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.584283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:55.586409 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 318/646] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:55.587230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.594551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 329/647] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:55.594551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.704962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a6312-a8a1-45b8-9809-d0af32d17c92 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.711545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86c78f-0d98-4b41-854f-2fed406191d5 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.727470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:55.740857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.741405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.741687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.741950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.749779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.750423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.751088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.751359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.752326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:55.752513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.752674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.754009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.754115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:55.769794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:55.770858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:55.771775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:55.771775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:55.771775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:55.772641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 319/648] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:55.773368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.774427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 330/649] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:55.775725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.835308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:55.888081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5cfb4-8f32-4b2d-a565-2554b1038e75 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.888288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cce4a-a269-40af-a9c1-3a2368f29bd3 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:55.925278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.925495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:55.925928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.926222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:55.926336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.926504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.926751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:55.927127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:55.927921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:55.928293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.929709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.938284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:55.939844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:55.947312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 320/650] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:55.951647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:55.955996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:55.957032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:55.957955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:55.957955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:55.957955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:55.957955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 331/651] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:55.957955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:55.987378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:56.063051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:56.065978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8090b6c-ce26-4da4-87e6-4dbfe64af3ce None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.070790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa862c3-f63c-490a-8610-63c3617a3afb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.102901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.103438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.103719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.103995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.104648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:56.104959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:56.106363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:56.108551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.109213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.109496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.109796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.110561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=c24d70e571c64f018aa199c4c08d153e) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:56.110871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:56.112299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:56.118290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 321/652] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/role_assignments?user.id=c24d70e571c64f018aa199c4c08d153e&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:56.119098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:56.120450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 332/653] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:56.121755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:56.241796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cbc59-852c-4642-9d00-b59649dd9e60 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.248806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:56.258968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cf2d5-8f89-480d-a281-ae1bc7864ceb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.283556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.284224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.284503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.284828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.285497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:56.285803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:56.287741 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:56.308108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 333/654] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:56.308108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:56.312528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fc483-8e0f-4aa7-84f6-dd93a81ce36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.313818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fc483-8e0f-4aa7-84f6-dd93a81ce36a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.313818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fc483-8e0f-4aa7-84f6-dd93a81ce36a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.313818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fc483-8e0f-4aa7-84f6-dd93a81ce36a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.337148 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:56.401973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fc483-8e0f-4aa7-84f6-dd93a81ce36a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:56.403535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fc483-8e0f-4aa7-84f6-dd93a81ce36a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:56.431989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8378530-55c1-4828-a2a5-bc0f67f3639d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.478952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.479585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.479863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.480140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.491662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:56.493384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:56.508693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:56.509702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 334/655] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:56.510662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:56.527180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:56.535771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 322/656] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:56.541113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fc483-8e0f-4aa7-84f6-dd93a81ce36a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:56.629681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:56.659754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bfd06-497e-4a52-8dbe-aaa634823524 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.666936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fc483-8e0f-4aa7-84f6-dd93a81ce36a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.705631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.706444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.706945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.707184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.708002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:56.708402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:56.709913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:56.717335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.718593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.719034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.719459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.730562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:56.732928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:56.733929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 323/657] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 13:33:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:56.734982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:56.749377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:56.750619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:33:56.751790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:56.754281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 335/658] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:56.755294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:56.803518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:56.861473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd329-9431-4462-bd6c-8b8e6ebac4ff None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.897145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.897831 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.898213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.898599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.904641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af0b2e-0b12-4f8e-bac9-fcdfc1709949 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:56.910840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:56.912296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:56.928574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:56.929604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:56.930536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:56.930536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:56.930536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:56.934407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 324/659] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:56.935977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:56.958965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18981-df6b-485e-9792-59b8f4bdb005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:56.959888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18981-df6b-485e-9792-59b8f4bdb005 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:56.960563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18981-df6b-485e-9792-59b8f4bdb005 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:56.961426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18981-df6b-485e-9792-59b8f4bdb005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:56.977223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:57.063374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbf161-1d75-42c3-9437-f7dbce409676 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:57.069445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18981-df6b-485e-9792-59b8f4bdb005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.070171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:57.070380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf18981-df6b-485e-9792-59b8f4bdb005 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.109603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:57.110297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.110578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.110850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.111627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:57.112027 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.113634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.132806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 325/660] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 8 13:33:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:57.133641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:57.195948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 336/661] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:57.197122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5e3d7-25bc-427a-abae-d29129d90056 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.197481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5e3d7-25bc-427a-abae-d29129d90056 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.197676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5e3d7-25bc-427a-abae-d29129d90056 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.202347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 337/662] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:57.202347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5e3d7-25bc-427a-abae-d29129d90056 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:57.245309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:57.247194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8d35-70db-424d-bc93-9c40d878b532 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:57.283052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:57.283660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.283939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.284211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.284846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:57.285221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.286637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.298731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 326/663] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:57.300174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:57.307445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5e3d7-25bc-427a-abae-d29129d90056 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:57.327908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:57.344839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f823e-79aa-4e73-b604-c9556bbc5aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:57.345445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f823e-79aa-4e73-b604-c9556bbc5aaa None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.346001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f823e-79aa-4e73-b604-c9556bbc5aaa None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.346001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f823e-79aa-4e73-b604-c9556bbc5aaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.405042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea801611-15b4-4e67-b05a-be39e0205d3c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:57.424490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f823e-79aa-4e73-b604-c9556bbc5aaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.426287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f823e-79aa-4e73-b604-c9556bbc5aaa None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.441537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb40c-bfcb-479d-a441-f9c8862f96ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:57.442183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb40c-bfcb-479d-a441-f9c8862f96ff None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.442466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb40c-bfcb-479d-a441-f9c8862f96ff None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.442736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb40c-bfcb-479d-a441-f9c8862f96ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.453784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbb40c-bfcb-479d-a441-f9c8862f96ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.455184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbb40c-bfcb-479d-a441-f9c8862f96ff None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-71cbb40c-bfcb-479d-a441-f9c8862f96ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:57.470799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:57.471973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:57.472969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:57.472969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:57.472969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:57.472969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 327/664] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:57.473213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26dfb-22ca-41d4-b02b-4a7326e166cf None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.473285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26dfb-22ca-41d4-b02b-4a7326e166cf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.473543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26dfb-22ca-41d4-b02b-4a7326e166cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.479350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:57.494619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-08d26dfb-22ca-41d4-b02b-4a7326e166cf None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:57.495813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-08d26dfb-22ca-41d4-b02b-4a7326e166cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:57.548920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 338/665] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:33:57.549823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f823e-79aa-4e73-b604-c9556bbc5aaa None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:57.574650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 328/666] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:57.574650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26dfb-22ca-41d4-b02b-4a7326e166cf None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:57.577337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:57.677376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f823e-79aa-4e73-b604-c9556bbc5aaa None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:57.713413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26dfb-22ca-41d4-b02b-4a7326e166cf None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:57.720137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:57.720761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.721036 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.721305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.730024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.732172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.735955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:57.747483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:57.754088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 339/667] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:57.755124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:57.762649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:57.763971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.764266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.764600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.765262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:57.765969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.767835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.785942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 329/668] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:57.787198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:57.861359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:57.877703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a47f23-0baf-4756-8feb-7e63c1baff4c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:57.900478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299cba5-c9a6-464d-ab38-fc9ae5b4e2c9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:57.921753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:57.922412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.922691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.923310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.934286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.935753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.938841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:57.939457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:57.939741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:57.940072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:57.940776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:57.941086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:57.942533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:57.950238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:57.952166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:57.953190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:57.953190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:57.953190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:57.953190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 340/669] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:57.953190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:57.968077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 330/670] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:57.968077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.007432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:58.059072 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086afcd6-207d-4a11-95f2-30ee6f2d36ec None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.074893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488ee6d-9a3f-468e-bcae-c631cd125e55 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.095139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.096217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.096739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.097247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.105287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:58.106912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:58.109784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:58.119204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:58.126858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 341/671] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:58.127786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.128388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95d044-050b-4b14-8f8e-2f0de1c1a21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.128388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d044-050b-4b14-8f8e-2f0de1c1a21c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.128388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d044-050b-4b14-8f8e-2f0de1c1a21c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.128388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d044-050b-4b14-8f8e-2f0de1c1a21c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.139384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95d044-050b-4b14-8f8e-2f0de1c1a21c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-8c95d044-050b-4b14-8f8e-2f0de1c1a21c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:58.144276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:58.145316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 331/672] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:33:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:58.147194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95d044-050b-4b14-8f8e-2f0de1c1a21c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.238235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a6576-3fda-4831-8981-c97c6d532626 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.253880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:58.266798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95d044-050b-4b14-8f8e-2f0de1c1a21c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.278762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.280017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.280637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.281220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.282388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:58.282902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:58.285453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:58.306132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 342/673] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:58.307692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.309756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.311012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.311299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.311585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.312364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=6c07c85b63514d10bb17b836c60f2824) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:58.312787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:58.314388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:58.330519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 332/674] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:58 2022] GET /identity/v3/role_assignments?user.id=6c07c85b63514d10bb17b836c60f2824&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:58.332465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.419133 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:58.466477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed970b-23b2-4a01-b023-26a2dc384a79 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.498527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5a93-86ba-42a1-89e0-ecb5a2815291 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.509922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.510690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.510930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.512111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.512539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:58.514097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:58.515061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:58.532757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 343/675] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:58.534056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.535989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c70ce-aa97-4007-9b3f-7dfed51b4ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.536648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c70ce-aa97-4007-9b3f-7dfed51b4ae7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.536930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c70ce-aa97-4007-9b3f-7dfed51b4ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.537310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c70ce-aa97-4007-9b3f-7dfed51b4ae7 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.537992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c70ce-aa97-4007-9b3f-7dfed51b4ae7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:58.538629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c70ce-aa97-4007-9b3f-7dfed51b4ae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:58.540582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c70ce-aa97-4007-9b3f-7dfed51b4ae7 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:58.554799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 333/676] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:33:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:58.556064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b1150-c4b5-49e3-8af3-1ea92893858e None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.556352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b1150-c4b5-49e3-8af3-1ea92893858e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.556628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b1150-c4b5-49e3-8af3-1ea92893858e None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.557846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 334/677] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:33:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:33:58.563714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b1150-c4b5-49e3-8af3-1ea92893858e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.581209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:58.668721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bda631-abd8-4f02-a526-52388f96d353 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.704773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b1150-c4b5-49e3-8af3-1ea92893858e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.705297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06443291-068d-4032-946b-1781d2451694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.705395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06443291-068d-4032-946b-1781d2451694 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.705685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06443291-068d-4032-946b-1781d2451694 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.705970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06443291-068d-4032-946b-1781d2451694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.713038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06443291-068d-4032-946b-1781d2451694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:58.714617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06443291-068d-4032-946b-1781d2451694 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:58.717154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-06443291-068d-4032-946b-1781d2451694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:33:58.730261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:58.737252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 344/678] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:33:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bdb39-b878-47c0-825e-a3dc2deb8949 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.738171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bdb39-b878-47c0-825e-a3dc2deb8949 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.739010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bdb39-b878-47c0-825e-a3dc2deb8949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.753067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.753436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-215bdb39-b878-47c0-825e-a3dc2deb8949 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:33:58.753635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.754225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.754793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-215bdb39-b878-47c0-825e-a3dc2deb8949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:33:58.754958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=817a190d001b40a2afee944f19270aee) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:58.755271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:58.756690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:58.773646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 335/679] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:58 2022] GET /identity/v3/role_assignments?user.id=817a190d001b40a2afee944f19270aee&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:58.782306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.861139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 345/680] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:33:58.862417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bdb39-b878-47c0-825e-a3dc2deb8949 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:58.924150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6be24-13ae-481a-b755-7b296abefbbe None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:58.931913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:58.969309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:58.970464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:58.970892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:58.971330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:58.987166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:58.989199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:59.019377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:59.020386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:59.021587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:59.021587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:59.021587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:59.021587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 336/681] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:59.027579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:59.064480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bdb39-b878-47c0-825e-a3dc2deb8949 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:59.073609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:59.110768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:59.111545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:59.112008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:59.112457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:59.113507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:59.113813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:59.115415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:59.131870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 346/682] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:33:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:59.139185 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:59.187283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094154b0-38b1-4689-b716-df225d559a1e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:59.233299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:59.236536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:59.236920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:59.237295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:59.238056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:59.239672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:59.241211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:59.247897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:59.260672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 337/683] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:59.266625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:59.278403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be37-bb7a-434d-a26e-bc04bbbefc92 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:59.315922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:59.316887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:59.317396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:59.317871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:59.318887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=a71d353c1bd849028930261e9306fa01) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:59.319450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:59.321088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:59.328225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 347/684] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/role_assignments?user.id=a71d353c1bd849028930261e9306fa01&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:59.329499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:59.393319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:59.417260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594ac2-3b05-4928-bc48-e459a7e00e6e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:59.498129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e88f53-0a77-4d59-b475-736eaa2feec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:59.500205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3799-c883-49f3-9eac-b094d897eaaa None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:59.500411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e88f53-0a77-4d59-b475-736eaa2feec9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:59.500701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e88f53-0a77-4d59-b475-736eaa2feec9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:59.500981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e88f53-0a77-4d59-b475-736eaa2feec9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:59.523754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e88f53-0a77-4d59-b475-736eaa2feec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-92e88f53-0a77-4d59-b475-736eaa2feec9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:33:59.529422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:59.530440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:33:59.531317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:59.532621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 338/685] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:59.538926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e88f53-0a77-4d59-b475-736eaa2feec9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:59.585219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:59.585896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:59.586131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:59.586434 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] PATH_INFO: `/v3/users/swift` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:59.602644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:59.604202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:33:59.620132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:59.621310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:33:59.622341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:33:59.622895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 348/686] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:59.630526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:59.647102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:59.667574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e88f53-0a77-4d59-b475-736eaa2feec9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:59.719840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:59.721629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:59.722368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:59.722802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:59.723656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:59.724252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:59.726642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:59.737657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 339/687] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:59.744334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:59.759625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:59.764352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b1536-5970-4d72-a97c-1673c532489c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:59.802037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:59.802815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:59.803171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:59.803588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:59.804426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:33:59.804852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:59.807025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:59.828553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 349/688] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:33:59.833988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:33:59.874129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765666c-479d-4c10-9dab-8ed298402849 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:33:59.941314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:33:59.942104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:33:59.942518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:33:59.942890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:33:59.955139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:33:59.956742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:33:59.963158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:33:59.973285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:33:59.980087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:33:59.980991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:33:59.980991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:33:59.980991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:33:59.980991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 340/689] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:33:59.980991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:00.030573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067ff4a-2483-438f-bf94-c2f5cbc25987 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:00.096681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:00.096907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d148-2581-4f89-a824-126cd0e77bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:00.097373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d148-2581-4f89-a824-126cd0e77bf0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:00.097661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d148-2581-4f89-a824-126cd0e77bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:00.097939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d148-2581-4f89-a824-126cd0e77bf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:00.172550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ce878-e54c-4440-9b09-5804bd867ea5 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:00.193286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6d148-2581-4f89-a824-126cd0e77bf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:00.194892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6d148-2581-4f89-a824-126cd0e77bf0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:00.243505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:00.244129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:00.244409 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:00.244681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:00.245503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:00.245805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:00.247238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:00.261526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:00.271657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 341/690] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:00.277361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:00.327428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 350/691] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:33:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:34:00.328379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d148-2581-4f89-a824-126cd0e77bf0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:00.389319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153369d7-c487-48e2-8b8d-292b100f23e9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:00.414599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:00.434541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:00.435772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:00.436058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:00.436336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:00.451944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:00.453400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:00.475680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:00.476768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:00.477746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:00.478312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 342/692] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:34:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:00.490262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:00.538247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d148-2581-4f89-a824-126cd0e77bf0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:00.583599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:00.584237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:00.584513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:00.584825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:00.597210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:00.598749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:00.616427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:00.617445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:00.618397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:00.618397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:00.618397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:00.622373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 351/693] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:00.623773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:00.651798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:00.737557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3e0b3-9414-4fbe-840a-47c2b4317f40 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:00.777582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:00.778735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:00.779291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:00.779709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:00.780510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:00.780974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:00.781639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051654ba-8dae-458b-837d-b90581e58451 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:00.783017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:00.806242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 352/694] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:00.807273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:00.854315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:00.856168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:00.857271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:00.857753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:00.859752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:00.860282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:00.864138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:00.904219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 343/695] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:34:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:00.909629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:00.962561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:00.965214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:00.995851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6c009-01b3-4b2e-a696-ad1a4dde3cbf None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:01.026352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1239a6f-b850-4829-9375-14b889707cc7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:01.043641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e009c5-6c1b-46f5-8601-b704a13cd378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:01.044888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e009c5-6c1b-46f5-8601-b704a13cd378 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.045539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e009c5-6c1b-46f5-8601-b704a13cd378 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.046044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e009c5-6c1b-46f5-8601-b704a13cd378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.053351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e009c5-6c1b-46f5-8601-b704a13cd378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:01.055106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e009c5-6c1b-46f5-8601-b704a13cd378 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:01.063939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7c79d-b535-4944-bb36-791a700ab24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:01.064560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7c79d-b535-4944-bb36-791a700ab24d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.064839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7c79d-b535-4944-bb36-791a700ab24d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.065115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7c79d-b535-4944-bb36-791a700ab24d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-f6e009c5-6c1b-46f5-8601-b704a13cd378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:01.068578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:01.069459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:01.070378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:01.070378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:01.070378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:01.070378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:01.070378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:01.070378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:01.070378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:01.070378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:01.077625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 353/696] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:01.079193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a78ca-0da5-4f2b-9fbd-ec303f656d69 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.079676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a78ca-0da5-4f2b-9fbd-ec303f656d69 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.080169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a78ca-0da5-4f2b-9fbd-ec303f656d69 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.082090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 354/697] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:01.083014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a78ca-0da5-4f2b-9fbd-ec303f656d69 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:01.149630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7c79d-b535-4944-bb36-791a700ab24d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:01.151115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7c79d-b535-4944-bb36-791a700ab24d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:01.194277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a78ca-0da5-4f2b-9fbd-ec303f656d69 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:01.220506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:01.244599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa173c-1739-4b12-9e33-7725df8afec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:01.245366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa173c-1739-4b12-9e33-7725df8afec5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.245638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa173c-1739-4b12-9e33-7725df8afec5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.245911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa173c-1739-4b12-9e33-7725df8afec5 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.246645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa173c-1739-4b12-9e33-7725df8afec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:01.246948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa173c-1739-4b12-9e33-7725df8afec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:01.248950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa173c-1739-4b12-9e33-7725df8afec5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:01.252011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:01.267981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 355/698] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:01.269344 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f97d0d-c93a-4553-97a7-ba7f15dbbdb6 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.269633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f97d0d-c93a-4553-97a7-ba7f15dbbdb6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.269906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f97d0d-c93a-4553-97a7-ba7f15dbbdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.289333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f2f97d0d-c93a-4553-97a7-ba7f15dbbdb6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:01.290724 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f2f97d0d-c93a-4553-97a7-ba7f15dbbdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:01.296024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 344/699] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:34:01.297663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7c79d-b535-4944-bb36-791a700ab24d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:01.377740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 356/700] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:34:01.377740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f97d0d-c93a-4553-97a7-ba7f15dbbdb6 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:01.474959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7c79d-b535-4944-bb36-791a700ab24d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:01.511709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f97d0d-c93a-4553-97a7-ba7f15dbbdb6 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:01.517944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9542a-672e-41b5-9b7b-151c2deddc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:01.518630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9542a-672e-41b5-9b7b-151c2deddc1e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.518918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9542a-672e-41b5-9b7b-151c2deddc1e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.519714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9542a-672e-41b5-9b7b-151c2deddc1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.520451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9542a-672e-41b5-9b7b-151c2deddc1e None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=136ce464d3194f98952beed2d89b8bc3) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:01.520760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9542a-672e-41b5-9b7b-151c2deddc1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:01.523091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9542a-672e-41b5-9b7b-151c2deddc1e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:01.530807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 345/701] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:01 2022] GET /identity/v3/role_assignments?user.id=136ce464d3194f98952beed2d89b8bc3&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:01.535342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc8e78-e16f-4c7d-b56d-859f9f05c2f0 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.535871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc8e78-e16f-4c7d-b56d-859f9f05c2f0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.536437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc8e78-e16f-4c7d-b56d-859f9f05c2f0 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.537972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 346/702] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:01.542645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:01.546060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f230e-8d0c-44d1-b9e9-678665f88f92 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.546613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f230e-8d0c-44d1-b9e9-678665f88f92 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.546992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f230e-8d0c-44d1-b9e9-678665f88f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.553076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:01.554120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.554637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.555107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.566190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:01.568736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:01.570620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:01.588847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:01.609050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:01.609968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:01.609968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:01.609968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:01.609968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 357/703] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:01.609968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:01.610366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b63f230e-8d0c-44d1-b9e9-678665f88f92 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:01.610366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b63f230e-8d0c-44d1-b9e9-678665f88f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:01.726268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 347/704] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:34:01.726927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f230e-8d0c-44d1-b9e9-678665f88f92 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:01.781929 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad13395-fbd4-477a-9b73-abd043ff6ae5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:01.827124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:01.827841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.828196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.828478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.829137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:01.829531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:01.831093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:01.845115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 358/705] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:01.850924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:01.855369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f230e-8d0c-44d1-b9e9-678665f88f92 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:01.893310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:01.899768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:01.905484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:01.905484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:01.905484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:01.908159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:01.915245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:01.916939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:01.935611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:01.936617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:01.937561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:01.937561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:01.937561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:01.940254 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 348/706] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:01.945936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:01.990425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d711123-c2a5-4b12-b029-a7f42e94c63c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:02.041244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:02.041937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.042195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.042472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.051369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:02.053399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:02.068192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:02.069167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:02.070068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 359/707] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:02.071073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:02.086480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de0ea9-1092-4ddf-a085-3e605cd02ca7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:02.127637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:02.128295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.128672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.128951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.129598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:02.129908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:02.131419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:02.150081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 349/708] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:02.150954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:02.187317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:02.191951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50b325-8193-4003-b691-3f92ab72ea15 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:02.208714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:02.227896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:02.228531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.228808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.229078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.229652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:02.231107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:02.242704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:02.243776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:02.244630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 360/709] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 8 13:34:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:02.245553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:02.272894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1b127-fe3b-49af-b23e-42a3672e9d0e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:02.314241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cc6b5-546b-4561-8678-ff0059dd070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:02.316004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cc6b5-546b-4561-8678-ff0059dd070c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.316004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cc6b5-546b-4561-8678-ff0059dd070c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.316004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cc6b5-546b-4561-8678-ff0059dd070c None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.316268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cc6b5-546b-4561-8678-ff0059dd070c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:02.316556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cc6b5-546b-4561-8678-ff0059dd070c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:02.318012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cc6b5-546b-4561-8678-ff0059dd070c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:02.337452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 350/710] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:02.337452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cf5e4-ea4e-4b96-966a-41c78554cc65 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.337452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cf5e4-ea4e-4b96-966a-41c78554cc65 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.337452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cf5e4-ea4e-4b96-966a-41c78554cc65 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.337452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 351/711] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:02.337452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cf5e4-ea4e-4b96-966a-41c78554cc65 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:02.361776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c5ead-244c-4865-b2d9-663a21e10f08 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:02.399345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ecaee-dfa1-41bd-828f-e92f57a87f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:02.399952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ecaee-dfa1-41bd-828f-e92f57a87f4a None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.400234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ecaee-dfa1-41bd-828f-e92f57a87f4a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.400502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ecaee-dfa1-41bd-828f-e92f57a87f4a None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.401480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ecaee-dfa1-41bd-828f-e92f57a87f4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:02.402729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ecaee-dfa1-41bd-828f-e92f57a87f4a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:02.460822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cf5e4-ea4e-4b96-966a-41c78554cc65 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:02.463104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:02.482603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 361/712] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:34:02 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:02.491560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38af73-0ee7-4276-9035-0e7dd89055e0 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.491884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38af73-0ee7-4276-9035-0e7dd89055e0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.492066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38af73-0ee7-4276-9035-0e7dd89055e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.496378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:02.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.497830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.498224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.498790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:02.499094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:02.500506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:02.511345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 352/713] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 13:34:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:02.511345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:02.531064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-be38af73-0ee7-4276-9035-0e7dd89055e0 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:02.532295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-be38af73-0ee7-4276-9035-0e7dd89055e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:02.548661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:02.628648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 362/714] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:34:02.631113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38af73-0ee7-4276-9035-0e7dd89055e0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:02.661812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1caa36-dba7-4fb0-8c83-cebe1fb5ffb2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:02.706261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:02.707269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.707559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.707829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.708580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=c24d70e571c64f018aa199c4c08d153e) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:02.708916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:02.710604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:02.721631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 353/715] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:02 2022] GET /identity/v3/role_assignments?user.id=c24d70e571c64f018aa199c4c08d153e&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=97f11873550e41dba0cd095afb1dec00 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:02.788966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38af73-0ee7-4276-9035-0e7dd89055e0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:02.833195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:02.833829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:02.834111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:02.834456 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:02.835107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:02.835413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:02.836905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:02.844104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 363/716] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 13:34:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:02.974636 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:03.021366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:03.103790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20d4d-afd5-4dee-9ce9-b887ddfdb0c6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:03.146698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:03.147330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:03.147608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:03.147877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] PATH_INFO: `/v3/services/network` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:03.148458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:03.149816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:03.163433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:03.164537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:03.166033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 354/717] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:02 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:03.171467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:03.188118 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:03.281241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94f4bb-3c08-4af7-8577-b381814c3163 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:03.330622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9eb3d-a9d0-42c8-b62e-96e208e0bbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:03.331248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9eb3d-a9d0-42c8-b62e-96e208e0bbd3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:03.331528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9eb3d-a9d0-42c8-b62e-96e208e0bbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:03.331798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9eb3d-a9d0-42c8-b62e-96e208e0bbd3 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:03.332439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9eb3d-a9d0-42c8-b62e-96e208e0bbd3 None admin] RBAC: Adding query filter params (name=network) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:03.332739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9eb3d-a9d0-42c8-b62e-96e208e0bbd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:03.334109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9eb3d-a9d0-42c8-b62e-96e208e0bbd3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:03.339456 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 364/718] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:03.345342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:03.496154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:03.504068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958a015-a49b-4c85-ab80-536794e89785 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:03.553870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bccf6-1b67-4c8c-a292-dc7bb8fe1b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:03.554505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bccf6-1b67-4c8c-a292-dc7bb8fe1b9a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:03.554782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bccf6-1b67-4c8c-a292-dc7bb8fe1b9a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:03.555051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bccf6-1b67-4c8c-a292-dc7bb8fe1b9a None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:03.555769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bccf6-1b67-4c8c-a292-dc7bb8fe1b9a None admin] RBAC: Adding query filter params (type=network) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:03.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bccf6-1b67-4c8c-a292-dc7bb8fe1b9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:03.557438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bccf6-1b67-4c8c-a292-dc7bb8fe1b9a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:03.565620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 355/719] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:04.457236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786d3fe-81b2-48c5-9db7-40666f602c2f None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:04.457760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786d3fe-81b2-48c5-9db7-40666f602c2f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:04.457760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786d3fe-81b2-48c5-9db7-40666f602c2f None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:04.459553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 365/720] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:04.464627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f06090-7dd7-49ac-93c0-59a12cfaece7 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:04.465053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f06090-7dd7-49ac-93c0-59a12cfaece7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:04.465251 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f06090-7dd7-49ac-93c0-59a12cfaece7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:04.491647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-94f06090-7dd7-49ac-93c0-59a12cfaece7 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:04.492867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-94f06090-7dd7-49ac-93c0-59a12cfaece7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:04.581794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 356/721] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:34:04.930546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786d3fe-81b2-48c5-9db7-40666f602c2f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:05.040700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:05.073902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786d3fe-81b2-48c5-9db7-40666f602c2f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:05.115192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eddef1-87e0-4162-82b0-02c2b46391e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:05.116073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eddef1-87e0-4162-82b0-02c2b46391e4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.116525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eddef1-87e0-4162-82b0-02c2b46391e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.116948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eddef1-87e0-4162-82b0-02c2b46391e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.130773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eddef1-87e0-4162-82b0-02c2b46391e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-35eddef1-87e0-4162-82b0-02c2b46391e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:05.139316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:05.140416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:05.141454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:05.147553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 366/722] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:05.150394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f06090-7dd7-49ac-93c0-59a12cfaece7 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:05.270610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:05.283347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f06090-7dd7-49ac-93c0-59a12cfaece7 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:05.336586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fa4d9-350e-4370-9f51-f745d96ce40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:05.337513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fa4d9-350e-4370-9f51-f745d96ce40d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.337944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fa4d9-350e-4370-9f51-f745d96ce40d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.338395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fa4d9-350e-4370-9f51-f745d96ce40d None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.339267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fa4d9-350e-4370-9f51-f745d96ce40d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:05.339727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fa4d9-350e-4370-9f51-f745d96ce40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:05.341352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fa4d9-350e-4370-9f51-f745d96ce40d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:05.354296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 357/723] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:05.360617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eddef1-87e0-4162-82b0-02c2b46391e4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:05.442785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cff04-9f5c-47c2-9300-996849bd066f None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.443075 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cff04-9f5c-47c2-9300-996849bd066f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.443346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cff04-9f5c-47c2-9300-996849bd066f None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.445018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 358/724] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:05.453799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907fa82-5e64-4f8d-9536-55f133fdbd7f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.453799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907fa82-5e64-4f8d-9536-55f133fdbd7f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.453799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907fa82-5e64-4f8d-9536-55f133fdbd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.474750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b907fa82-5e64-4f8d-9536-55f133fdbd7f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:05.475985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b907fa82-5e64-4f8d-9536-55f133fdbd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:05.501519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eddef1-87e0-4162-82b0-02c2b46391e4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:05.564659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:05.565225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.565486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.565752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.570675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:05.587332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:05.588927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:05.596960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 359/725] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:34:05.598728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a343a01-9e9d-4b95-ab36-582c1dd45a22 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.599190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a343a01-9e9d-4b95-ab36-582c1dd45a22 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.599618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a343a01-9e9d-4b95-ab36-582c1dd45a22 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.605662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 360/726] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:05.607376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dabb01-2c69-4ac3-a58f-be9bef36bf97 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.607817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dabb01-2c69-4ac3-a58f-be9bef36bf97 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.608241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dabb01-2c69-4ac3-a58f-be9bef36bf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:05.617116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:05.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:05.619517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:05.619517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:05.619517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:05.622623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 367/727] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:05.629371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:05.642783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-94dabb01-2c69-4ac3-a58f-be9bef36bf97 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:05.644203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-94dabb01-2c69-4ac3-a58f-be9bef36bf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:05.742223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 361/728] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:34:05.744328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d45e99-acdc-411a-b975-12e92787ebd3 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.744986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d45e99-acdc-411a-b975-12e92787ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.746033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d45e99-acdc-411a-b975-12e92787ebd3 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.748195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 362/729] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:05.753692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6eea-b1bb-4e31-ade5-b9682622b3c0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.754159 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6eea-b1bb-4e31-ade5-b9682622b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.754635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6eea-b1bb-4e31-ade5-b9682622b3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.789947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-515f6eea-b1bb-4e31-ade5-b9682622b3c0 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:05.792666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-515f6eea-b1bb-4e31-ade5-b9682622b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:05.814969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:05.827403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078612-a6cb-4040-bb4c-f4fa733ad776 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:05.867430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:05.868339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:05.868832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:05.869291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:05.870212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:05.870702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:05.873354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:05.886802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 368/730] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:05.894151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:05.939914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 363/731] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:34:05.940866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6eea-b1bb-4e31-ade5-b9682622b3c0 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:05.958215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:06.032732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b2a3-41c8-4559-a621-1301c84ab077 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.080298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6eea-b1bb-4e31-ade5-b9682622b3c0 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.085014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.085745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.086095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.086460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] PATH_INFO: `/v3/users/swift` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.095086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:06.096643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:06.109729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:06.110736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:06.111677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 369/732] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:06.119039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:06.149128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae53ea2-10ba-4770-8f92-84e310335a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.150246 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53ea2-10ba-4770-8f92-84e310335a24 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.150581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53ea2-10ba-4770-8f92-84e310335a24 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.150809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53ea2-10ba-4770-8f92-84e310335a24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.163310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae53ea2-10ba-4770-8f92-84e310335a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-cae53ea2-10ba-4770-8f92-84e310335a24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:06.169141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:06.170084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:06.170961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 364/733] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:06.171657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae53ea2-10ba-4770-8f92-84e310335a24 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:06.180446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:06.258811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a08048-858f-4d79-a027-5daf0693aefe None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.269446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:06.290273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae53ea2-10ba-4770-8f92-84e310335a24 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.294656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.295262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.295611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.295851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.296970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:06.297638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:06.299671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:06.317925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 370/734] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 13:34:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:06.318769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:06.330957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299486a4-a3a0-41be-88fe-fbd85a6d5b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.331500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299486a4-a3a0-41be-88fe-fbd85a6d5b88 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.331785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299486a4-a3a0-41be-88fe-fbd85a6d5b88 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.332269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299486a4-a3a0-41be-88fe-fbd85a6d5b88 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.332707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299486a4-a3a0-41be-88fe-fbd85a6d5b88 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:06.334106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299486a4-a3a0-41be-88fe-fbd85a6d5b88 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:06.427996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd93c6-e117-4150-86b0-3cd29c68e2f8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.455485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 365/735] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:34:06 2022] POST /identity/v3/services => generated 251 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:06.455485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299486a4-a3a0-41be-88fe-fbd85a6d5b88 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:06.472742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.473369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.473647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.473915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.474626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:06.474928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:06.476329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:06.486131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 371/736] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:06.487024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:06.520075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:06.533255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:06.577070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299486a4-a3a0-41be-88fe-fbd85a6d5b88 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.608397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86123a7-1b03-4f05-88a8-c812c66dd0b4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.620390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d2e53-0c19-4ced-a6a7-00caf6bc3d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.621048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d2e53-0c19-4ced-a6a7-00caf6bc3d7c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.621872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d2e53-0c19-4ced-a6a7-00caf6bc3d7c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.622169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d2e53-0c19-4ced-a6a7-00caf6bc3d7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.635024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d2e53-0c19-4ced-a6a7-00caf6bc3d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-660d2e53-0c19-4ced-a6a7-00caf6bc3d7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:06.640339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:06.641460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 366/737] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:34:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:06.642543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809a1c-510d-40e8-bbb3-e89947dc10d3 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.643807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809a1c-510d-40e8-bbb3-e89947dc10d3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.643807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809a1c-510d-40e8-bbb3-e89947dc10d3 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.645560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 367/738] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:06.645560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13809a1c-510d-40e8-bbb3-e89947dc10d3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:06.658346 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ab8cd-14c1-4716-8fec-c84fd973e912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.658971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ab8cd-14c1-4716-8fec-c84fd973e912 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.659253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ab8cd-14c1-4716-8fec-c84fd973e912 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.659531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ab8cd-14c1-4716-8fec-c84fd973e912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.733963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ab8cd-14c1-4716-8fec-c84fd973e912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:06.735489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ab8cd-14c1-4716-8fec-c84fd973e912 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:06.762044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:06.764234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13809a1c-510d-40e8-bbb3-e89947dc10d3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.788119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:06.801976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.802653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.802939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.803216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.814550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:06.816094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:06.832432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:06.833625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:06.834957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:06.834957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:06.834957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:06.838440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 368/739] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:06.839182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:06.851862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 372/740] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:06 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:34:06.857973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358facee-0760-49e9-9b72-d3567586030f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.858355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358facee-0760-49e9-9b72-d3567586030f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:06.858761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358facee-0760-49e9-9b72-d3567586030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:06.877357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-358facee-0760-49e9-9b72-d3567586030f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:06.878598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-358facee-0760-49e9-9b72-d3567586030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:06.962163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 373/741] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:34:06.962371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd2b0-b94f-495f-82ba-e5891a1006f9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:06.962468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358facee-0760-49e9-9b72-d3567586030f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:06.998793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:06.999561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:06.999937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.000317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.001176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:07.001576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.003238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.010980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:07.013977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 369/742] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:34:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:07.014950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.054558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:07.057777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358facee-0760-49e9-9b72-d3567586030f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.102298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.102914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.103258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.103546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.104297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:07.104606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.106095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.121653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 374/743] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:07.122508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.136483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c9650-331f-4dfa-9124-7ef0d318de68 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.178048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.179082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.179580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.180051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.193698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.196248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:07.216918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:07.218553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:07.219453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:07.219453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:07.219453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:07.222232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 370/744] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:07.223487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.270867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc9e3-097c-4ed4-8923-8d29e4bf4685 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.289691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:07.310528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.313671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.313671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.313671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.322809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.324289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.325164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:07.338392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40eb61b-a2db-4646-a7a2-478a28249419 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:07.341355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:07.348384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:07.349287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:07.349287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:07.349287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:07.349287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 375/745] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:07.349287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.379258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.379902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.380190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.380467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.390125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.391691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:07.404613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:07.405676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 371/746] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:07.406763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.455156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299b01a-00c2-4cc7-a6c4-89dcdc08dc47 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.491763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.492049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.492321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.492957 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:07.493260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.494855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.506701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 376/747] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:07.511629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.516033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db2ff0-3f3f-4de7-9272-a74d74c0ff3a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.538749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:07.551515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac692fc-d1df-4f67-aec9-2d3b494f0cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.552347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac692fc-d1df-4f67-aec9-2d3b494f0cc4 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.554289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac692fc-d1df-4f67-aec9-2d3b494f0cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.554289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac692fc-d1df-4f67-aec9-2d3b494f0cc4 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.554289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac692fc-d1df-4f67-aec9-2d3b494f0cc4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.555020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac692fc-d1df-4f67-aec9-2d3b494f0cc4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.615408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da53d24-998a-454b-a6b7-a6863102ee30 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.640685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 372/748] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:34:07 2022] POST /identity/v3/services => generated 240 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:07.647216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac692fc-d1df-4f67-aec9-2d3b494f0cc4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.660361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.661351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.661811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.662288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.664018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:07.664579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.666301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:07.667762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.687755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 377/749] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:07.688807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.794689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac692fc-d1df-4f67-aec9-2d3b494f0cc4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.812582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:07.836767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.837382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.837658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.838129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.838933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af837e76-54b4-4efe-a159-3de8ec706a03 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:07.839066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:07.839066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.841025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.862302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 373/750] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:07.863299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.878752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:07.879386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:07.879664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:07.879934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:07.891658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:07.891658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:07.908075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:07.909021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:07.910521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 378/751] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:07.914326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:07.945192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:07.988820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636be806-c6ad-4ea9-8087-3be0a9dea7bf None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:08.029951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef0ed0-7aba-46c4-8c3f-48622f5c9df5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:08.030630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:08.031274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.031557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.031833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.043314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:08.044865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:08.059152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:08.060334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:08.061859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:08.062490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 374/752] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:08.065094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:08.080314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:08.080956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.081396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.081680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.095880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:08.097897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:08.100631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:08.118339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:08.120455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:08.122530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:08.122530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:08.122530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:08.122530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 379/753] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:08.122530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:08.184946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17787-fc4c-4c78-bb85-138b666f6634 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:08.215885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:08.225685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:08.226603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.228273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.228273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.228273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:08.228835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:08.231713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:08.248755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 375/754] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:08.249916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:08.262014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597cbe2-0302-41f7-9a5f-b60413b64fe2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:08.307586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:08.308302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.308673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.309322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.310095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:08.310685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:08.312285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:08.338374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 380/755] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 13:34:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:08.339593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:08.377006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180782-0374-467f-8d64-8ccd6fe66b76 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:08.416471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:08.418009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:08.418842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.419269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.419688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.420957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:08.421417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:08.423423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:08.438477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 376/756] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:08.439627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:08.475784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0079a0-e2e0-4bc8-8229-c32433c6c4c1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:08.503270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:08.519156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:08.522382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.522382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.522382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.522382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=136ce464d3194f98952beed2d89b8bc3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:08.522382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:08.522944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:08.537229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 381/757] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:08 2022] GET /identity/v3/role_assignments?user.id=136ce464d3194f98952beed2d89b8bc3&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=97f11873550e41dba0cd095afb1dec00 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:08.538289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:08.570718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4aaa14-5fb2-4875-8bf8-34821d2bccdd None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:08.618237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d274b-a45c-4a8a-8a60-a815d5584e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:08.619776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d274b-a45c-4a8a-8a60-a815d5584e76 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.620234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d274b-a45c-4a8a-8a60-a815d5584e76 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.620672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d274b-a45c-4a8a-8a60-a815d5584e76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.708150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d274b-a45c-4a8a-8a60-a815d5584e76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:08.709874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d274b-a45c-4a8a-8a60-a815d5584e76 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:08.744996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1171c8-95b7-4e9b-bd76-70352883a8c8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:08.777785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:08.779321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:08.801556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a7bef-bea2-402b-a0c7-db75d78b2675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:08.802860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a7bef-bea2-402b-a0c7-db75d78b2675 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.803155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a7bef-bea2-402b-a0c7-db75d78b2675 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.803432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a7bef-bea2-402b-a0c7-db75d78b2675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.815230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a7bef-bea2-402b-a0c7-db75d78b2675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:08.816718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a7bef-bea2-402b-a0c7-db75d78b2675 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-439a7bef-bea2-402b-a0c7-db75d78b2675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:08.831349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:08.832489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:08.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 382/758] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:08.834254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189735-b666-4366-a535-dd4fde08194c None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.834254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189735-b666-4366-a535-dd4fde08194c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.834382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189735-b666-4366-a535-dd4fde08194c None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.835691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 383/759] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:08.840456 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75189735-b666-4366-a535-dd4fde08194c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:08.865704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 377/760] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:08.867992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977e0c2-de72-47b7-8605-b0b2d3a0e8af None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:08.868433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977e0c2-de72-47b7-8605-b0b2d3a0e8af None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:08.868855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977e0c2-de72-47b7-8605-b0b2d3a0e8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:08.889689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b977e0c2-de72-47b7-8605-b0b2d3a0e8af None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:08.891637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b977e0c2-de72-47b7-8605-b0b2d3a0e8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:08.971738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 378/761] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:08.972860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977e0c2-de72-47b7-8605-b0b2d3a0e8af None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:08.979677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75189735-b666-4366-a535-dd4fde08194c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:09.059720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:09.070926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:09.072837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:09.072837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:09.072837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:09.072837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:09.073790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:09.075658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:09.087123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977e0c2-de72-47b7-8605-b0b2d3a0e8af None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:09.112644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 384/762] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:09.118096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:09.135879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:09.137370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:09.137802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:09.138291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:09.152643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:09.154465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:09.174793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:09.176123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:09.177226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:09.177226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:09.177226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:09.179116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 379/763] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:09.185949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:09.209734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:09.332388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77edc04-f39e-4a1d-95fc-8838fcd46803 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:09.359230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2b477-dfa3-4e5f-ba52-12cbd73263c1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:09.362712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:09.371730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:09.381792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:09.381792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:09.381792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:09.381792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:09.381792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:09.381792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:09.390881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 380/764] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:09.391996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:09.398653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f6216-7f03-4823-b964-d25fbc787944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:09.399461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f6216-7f03-4823-b964-d25fbc787944 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:09.399834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f6216-7f03-4823-b964-d25fbc787944 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:09.400204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f6216-7f03-4823-b964-d25fbc787944 None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/c24d70e571c64f018aa199c4c08d153e/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:09.403666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f6216-7f03-4823-b964-d25fbc787944 None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=c24d70e571c64f018aa199c4c08d153e, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:09.405472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f6216-7f03-4823-b964-d25fbc787944 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:09.490381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 385/765] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:34:09 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/c24d70e571c64f018aa199c4c08d153e/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:34:09.491679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f6216-7f03-4823-b964-d25fbc787944 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:09.502649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d5d3d-d17d-4ed7-837d-85229e8f7759 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:09.543264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:09.544262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:09.544686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:09.545122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:09.545873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:09.547574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:09.561932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:09.563088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:09.563999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:09.566835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 381/766] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 8 13:34:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:09.576016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:09.650980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:09.701681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26313bf-29ee-4366-b0b9-41e2c28656b7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:09.748313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:09.749192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:09.750148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:09.750558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:09.751511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:09.751999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:09.754249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:09.756349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:09.763509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 382/767] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 13:34:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:09.764862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f6216-7f03-4823-b964-d25fbc787944 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:09.769107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:09.811090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:09.812821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:09.813291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:09.813733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:09.824530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:09.827017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:09.845572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:09.846758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:09.847866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:09.847866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:09.847866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:09.847866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:09.847866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:09.847866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:09.847866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:09.847866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:09.850305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 386/768] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:09.856498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:09.898700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca66f-c9e6-4ae9-95af-bb22cf7ffe74 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:09.921487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:09.954930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:09.955918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:09.956366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:09.956787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:09.957590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:09.958045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:09.959684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:09.968591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 383/769] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 13:34:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:09.975164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:09.983406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aed2d-b101-45ed-81cb-60bd0d294ed1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:10.021089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:10.021673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:10.021954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:10.022254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:10.023226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:10.023226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:10.024664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:10.037516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 387/770] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:10.043874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:10.064803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:10.116535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efae146-6bec-4d57-9330-95c4c116c3d1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:10.154725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4bbfb-0322-47c0-81fc-8f0ab6a1ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:10.155634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4bbfb-0322-47c0-81fc-8f0ab6a1ab2f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:10.156068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4bbfb-0322-47c0-81fc-8f0ab6a1ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:10.156521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4bbfb-0322-47c0-81fc-8f0ab6a1ab2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:10.157375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4bbfb-0322-47c0-81fc-8f0ab6a1ab2f None admin] RBAC: Adding query filter params (interface=public, service_id=3833a6d7837f4eb58e86e320c2a3c0a3, region_id=RegionOne) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:10.157833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4bbfb-0322-47c0-81fc-8f0ab6a1ab2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:10.159017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167c2c8-69da-4628-a799-d0d5e1e8cf6f None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:10.159579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4bbfb-0322-47c0-81fc-8f0ab6a1ab2f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:10.170523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 384/771] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 13:34:09 2022] GET /identity/v3/endpoints?service_id=3833a6d7837f4eb58e86e320c2a3c0a3&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:10.172119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e4aa5-fd79-44c9-aa3a-cd99d81210ff None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:10.172621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e4aa5-fd79-44c9-aa3a-cd99d81210ff None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:10.173107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e4aa5-fd79-44c9-aa3a-cd99d81210ff None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:10.174849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 385/772] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:10.176094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e4aa5-fd79-44c9-aa3a-cd99d81210ff None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:10.202186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978299e-aca5-4556-b263-c0b96220f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:10.202861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978299e-aca5-4556-b263-c0b96220f8bb None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:10.203269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978299e-aca5-4556-b263-c0b96220f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:10.203511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978299e-aca5-4556-b263-c0b96220f8bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:10.204247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978299e-aca5-4556-b263-c0b96220f8bb None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=df727be6307948c49affa47e601f989c) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:10.204880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978299e-aca5-4556-b263-c0b96220f8bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:10.206153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978299e-aca5-4556-b263-c0b96220f8bb None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:10.213656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 388/773] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:10 2022] GET /identity/v3/role_assignments?user.id=df727be6307948c49affa47e601f989c&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:10.218822 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24a9a5-eb22-44b2-9b99-77dce0345e22 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:10.218822 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24a9a5-eb22-44b2-9b99-77dce0345e22 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:10.218822 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24a9a5-eb22-44b2-9b99-77dce0345e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:10.239542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:10.250335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-cb24a9a5-eb22-44b2-9b99-77dce0345e22 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:10.251681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cb24a9a5-eb22-44b2-9b99-77dce0345e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:10.307079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e4aa5-fd79-44c9-aa3a-cd99d81210ff None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:10.405334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 389/774] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:10.480750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2302c2-a0ed-420c-b47e-5c76b91bc5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:10.481736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2302c2-a0ed-420c-b47e-5c76b91bc5f7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:10.482372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2302c2-a0ed-420c-b47e-5c76b91bc5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:10.482883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2302c2-a0ed-420c-b47e-5c76b91bc5f7 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:10.483703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2302c2-a0ed-420c-b47e-5c76b91bc5f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:10.485267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2302c2-a0ed-420c-b47e-5c76b91bc5f7 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:10.491473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 386/775] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:34:10 2022] GET /identity/v3/services => generated 772 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:10.711598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24a9a5-eb22-44b2-9b99-77dce0345e22 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:10.724313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:10.821736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24a9a5-eb22-44b2-9b99-77dce0345e22 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:10.862259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579c624-23cf-4275-8335-2b3a0713f2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:10.862904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579c624-23cf-4275-8335-2b3a0713f2a7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:10.863186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579c624-23cf-4275-8335-2b3a0713f2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:10.863463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579c624-23cf-4275-8335-2b3a0713f2a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:10.864053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579c624-23cf-4275-8335-2b3a0713f2a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:10.865435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579c624-23cf-4275-8335-2b3a0713f2a7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-9579c624-23cf-4275-8335-2b3a0713f2a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:10.875936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:10.876987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:10.877841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 390/776] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:10 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:10.883445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2302c2-a0ed-420c-b47e-5c76b91bc5f7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:11.055586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e264e1-322b-4f31-8978-8df01786ecb2 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:11.055876 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e264e1-322b-4f31-8978-8df01786ecb2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:11.056156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e264e1-322b-4f31-8978-8df01786ecb2 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:11.057968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 391/777] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:11.067459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afedbfe-d134-4519-ab0a-48fb2ef2b60b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:11.067738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afedbfe-d134-4519-ab0a-48fb2ef2b60b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:11.068033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afedbfe-d134-4519-ab0a-48fb2ef2b60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:11.073596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2302c2-a0ed-420c-b47e-5c76b91bc5f7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:11.084423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:11.102321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0afedbfe-d134-4519-ab0a-48fb2ef2b60b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:11.103604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0afedbfe-d134-4519-ab0a-48fb2ef2b60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:11.115806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:11.116734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:11.117224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:11.117673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:11.118499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] RBAC: Adding query filter params (name=network) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:11.119243 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:11.124558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:11.133114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 387/778] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:11.139299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:11.231343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 392/779] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:11.285674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb17d1-bb2f-4395-bff5-a711d43c55b0 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:11.359413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:11.360336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:11.360791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:11.361236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:11.362039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] RBAC: Adding query filter params (type=network) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:11.362571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:11.364099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:11.374788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 388/780] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:11.380903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afedbfe-d134-4519-ab0a-48fb2ef2b60b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:11.384639 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:11.497359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afedbfe-d134-4519-ab0a-48fb2ef2b60b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:11.538863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:11.539795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:11.540253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:11.540706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:11.541622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] RBAC: Adding query filter params (interface=public, service_id=4b94de6a23fc4298ae6be5a0fc9faf5c, region_id=RegionOne) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:11.542188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:11.544393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:11.557056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 393/781] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 13:34:11 2022] GET /identity/v3/endpoints?service_id=4b94de6a23fc4298ae6be5a0fc9faf5c&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:11.562457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:11.607254 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:11.649402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:11.704931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c943b-ce14-4d5f-b999-ac735bfd8538 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:11.727354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:11.773299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71300d-7c29-4f03-95f8-90854951eb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:11.774234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71300d-7c29-4f03-95f8-90854951eb29 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:11.774916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71300d-7c29-4f03-95f8-90854951eb29 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:11.775471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71300d-7c29-4f03-95f8-90854951eb29 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:11.777340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71300d-7c29-4f03-95f8-90854951eb29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:11.779759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71300d-7c29-4f03-95f8-90854951eb29 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:11.791115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 389/782] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:34:11 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:11.794523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d5dc9-22ac-4bfa-823e-7dca57d8beb2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:11.833960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01006ef6-80e3-4d74-9b0f-b8f42423c683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:11.835399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006ef6-80e3-4d74-9b0f-b8f42423c683 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:11.835825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006ef6-80e3-4d74-9b0f-b8f42423c683 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:11.836273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006ef6-80e3-4d74-9b0f-b8f42423c683 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:11.849704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01006ef6-80e3-4d74-9b0f-b8f42423c683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-01006ef6-80e3-4d74-9b0f-b8f42423c683 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:11.856027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:11.857174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:11.858231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:11.860601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 394/783] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:11.872047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71300d-7c29-4f03-95f8-90854951eb29 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:12.007158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:12.011796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71300d-7c29-4f03-95f8-90854951eb29 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:12.060811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:12.061426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:12.061705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:12.061975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:12.062683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:12.062979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:12.064329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:12.073222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 390/784] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:12.080192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01006ef6-80e3-4d74-9b0f-b8f42423c683 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:12.197178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01006ef6-80e3-4d74-9b0f-b8f42423c683 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:12.229070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:12.239159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3187b4d-c10a-46d8-ab64-870015e8feba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:12.239803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3187b4d-c10a-46d8-ab64-870015e8feba None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:12.240142 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3187b4d-c10a-46d8-ab64-870015e8feba None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:12.240416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3187b4d-c10a-46d8-ab64-870015e8feba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:12.251635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3187b4d-c10a-46d8-ab64-870015e8feba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:12.253419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3187b4d-c10a-46d8-ab64-870015e8feba None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-a3187b4d-c10a-46d8-ab64-870015e8feba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:12.273491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:12.274504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:12.275413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:12.275413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:12.275413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:12.275413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 395/785] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:12.281600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:12.345033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aef21-7691-4572-82f5-380f5fcce6e4 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:12.345408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aef21-7691-4572-82f5-380f5fcce6e4 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:12.345762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aef21-7691-4572-82f5-380f5fcce6e4 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:12.347473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 396/786] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:12.352515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a90ba-9c7e-4cd3-90dc-80bc0637434a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:12.352868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a90ba-9c7e-4cd3-90dc-80bc0637434a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:12.353255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a90ba-9c7e-4cd3-90dc-80bc0637434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:12.376145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-460a90ba-9c7e-4cd3-90dc-80bc0637434a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:12.377337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-460a90ba-9c7e-4cd3-90dc-80bc0637434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:12.469226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2148a-8ffa-4043-a4f8-49d24cd05ef3 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:12.520732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 397/787] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:12.533360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:12.534297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:12.534767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:12.535224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:12.536061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:12.536549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:12.538117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:12.557693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 391/788] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:12.563581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a90ba-9c7e-4cd3-90dc-80bc0637434a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:12.707145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a90ba-9c7e-4cd3-90dc-80bc0637434a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:12.724945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:12.757040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:12.757737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:12.758018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:12.758317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:12.769616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:12.771206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:12.787478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:12.788491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:12.789439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 398/789] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:12.796004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:12.834951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:12.836796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:12.954165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1dc3a-9c1d-4a9a-b64d-8d30ab57b7a8 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:12.982760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3090dc1-f2e6-4eea-87c2-aaf1c0290ae6 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:13.024758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.025407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.025640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.025909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.026870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:13.027188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:13.028598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:13.056027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c60bdf-f5d8-421e-a067-e625786db3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.056601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60bdf-f5d8-421e-a067-e625786db3c9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.056876 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60bdf-f5d8-421e-a067-e625786db3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.057146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60bdf-f5d8-421e-a067-e625786db3c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.067507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 392/790] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 13:34:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:13.075341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.083743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c60bdf-f5d8-421e-a067-e625786db3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-e8c60bdf-f5d8-421e-a067-e625786db3c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:13.089785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:13.090967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:13.092127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:13.096442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 399/791] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:13.099362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c60bdf-f5d8-421e-a067-e625786db3c9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.149394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:13.220822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:13.248583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28292ac-6d9b-4dba-98f8-fa68c27cf39b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:13.249840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c60bdf-f5d8-421e-a067-e625786db3c9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:13.286114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709e32c-f9a8-4058-bb9e-3c01f24109d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.286755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709e32c-f9a8-4058-bb9e-3c01f24109d8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.287035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709e32c-f9a8-4058-bb9e-3c01f24109d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.287304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709e32c-f9a8-4058-bb9e-3c01f24109d8 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.287959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709e32c-f9a8-4058-bb9e-3c01f24109d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:13.288257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709e32c-f9a8-4058-bb9e-3c01f24109d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:13.289935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709e32c-f9a8-4058-bb9e-3c01f24109d8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:13.292216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1589862-2e1c-42a1-accb-c8f1c618ddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.294021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589862-2e1c-42a1-accb-c8f1c618ddda None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.294537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589862-2e1c-42a1-accb-c8f1c618ddda None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.294985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589862-2e1c-42a1-accb-c8f1c618ddda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.300658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 400/792] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:13.301965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb416-248c-4e09-9fa5-224d53865850 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.302299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb416-248c-4e09-9fa5-224d53865850 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.302575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb416-248c-4e09-9fa5-224d53865850 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.303979 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 401/793] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:13.307758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1589862-2e1c-42a1-accb-c8f1c618ddda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:13.308751 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eb416-248c-4e09-9fa5-224d53865850 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.309690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1589862-2e1c-42a1-accb-c8f1c618ddda None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-d1589862-2e1c-42a1-accb-c8f1c618ddda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:13.328697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:13.329742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:13.330682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:13.330682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:13.330682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:13.331551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 393/794] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:13.334707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a580ab-456e-4e6a-b043-c57b47d0f1f3 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.335757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a580ab-456e-4e6a-b043-c57b47d0f1f3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.336237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a580ab-456e-4e6a-b043-c57b47d0f1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.360830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-74a580ab-456e-4e6a-b043-c57b47d0f1f3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:13.362235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-74a580ab-456e-4e6a-b043-c57b47d0f1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:13.411797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:13.421978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eb416-248c-4e09-9fa5-224d53865850 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:13.431922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 394/795] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:13.432772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a580ab-456e-4e6a-b043-c57b47d0f1f3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.467537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa257c1-ef7a-4c4c-b65f-e32fbbf8c93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.468093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa257c1-ef7a-4c4c-b65f-e32fbbf8c93b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.468369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa257c1-ef7a-4c4c-b65f-e32fbbf8c93b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.468689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa257c1-ef7a-4c4c-b65f-e32fbbf8c93b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.479173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa257c1-ef7a-4c4c-b65f-e32fbbf8c93b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:13.481209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa257c1-ef7a-4c4c-b65f-e32fbbf8c93b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-faa257c1-ef7a-4c4c-b65f-e32fbbf8c93b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:13.496529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:13.497536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:13.498497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:13.498497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:13.498497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:13.498497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 402/796] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:13.499211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6283ba5-9262-4571-b74c-7ca48f6fa869 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.499417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6283ba5-9262-4571-b74c-7ca48f6fa869 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.499687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6283ba5-9262-4571-b74c-7ca48f6fa869 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.504874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 403/797] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:13.504874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6283ba5-9262-4571-b74c-7ca48f6fa869 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.521409 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:13.576135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a580ab-456e-4e6a-b043-c57b47d0f1f3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:13.615245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6283ba5-9262-4571-b74c-7ca48f6fa869 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:13.622420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50027c1f-11ab-49db-8ef4-da37d43c9e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.623846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50027c1f-11ab-49db-8ef4-da37d43c9e43 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.624341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50027c1f-11ab-49db-8ef4-da37d43c9e43 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.624838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50027c1f-11ab-49db-8ef4-da37d43c9e43 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.626222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50027c1f-11ab-49db-8ef4-da37d43c9e43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:13.627174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50027c1f-11ab-49db-8ef4-da37d43c9e43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:13.629065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50027c1f-11ab-49db-8ef4-da37d43c9e43 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:13.644343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 395/798] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:13.649607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8631-55ed-4de3-acf4-48f402185681 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.649607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8631-55ed-4de3-acf4-48f402185681 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.649607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8631-55ed-4de3-acf4-48f402185681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.668279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.669248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.669787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.670352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.671294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:13.671494 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f2cd8631-55ed-4de3-acf4-48f402185681 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:13.671996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:13.672676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f2cd8631-55ed-4de3-acf4-48f402185681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:13.673840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:13.687856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 404/799] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:13.689146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.693265 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:13.750305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 396/800] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:13.750305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd8631-55ed-4de3-acf4-48f402185681 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.806520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:13.819763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bae77a-c841-4275-8607-ecc588cbf295 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:13.859547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd8631-55ed-4de3-acf4-48f402185681 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:13.867508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.869092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.869549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.869966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.879871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:13.882022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:13.895941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:13.897080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:13.898312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:13.898312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:13.898312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:13.898312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:13.898312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:13.898312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:13.898312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:13.898312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:13.899570 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 405/801] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:13.900503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.911674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:13.911674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:13.911674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:13.911674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] PATH_INFO: `/v3/users/nova` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:13.919685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:13.921229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:13.936192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:13.937382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:13.938405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 397/802] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:13.939007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:13.973630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:14.056968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c3277-6077-4459-b616-056598fcf1db None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.075200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:14.097710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.098397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.098675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.098947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.099645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:14.099943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:14.101363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:14.116643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 398/803] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:14.117424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:14.134789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfe4a4-0252-4b22-8973-f7b63796c598 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.170522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6abc0d-b61f-4170-b20c-cfb712b3fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.171153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6abc0d-b61f-4170-b20c-cfb712b3fc11 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.171427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6abc0d-b61f-4170-b20c-cfb712b3fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.171693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6abc0d-b61f-4170-b20c-cfb712b3fc11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.182967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6abc0d-b61f-4170-b20c-cfb712b3fc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-4f6abc0d-b61f-4170-b20c-cfb712b3fc11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:14.187768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:14.188726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:14.189573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 406/804] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:34:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:14.190325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6abc0d-b61f-4170-b20c-cfb712b3fc11 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:14.232708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab420f-fce5-4b33-a0f5-85c758cb0379 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.267471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.268379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.268659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.268937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.269714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:14.270021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:14.272443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:14.296896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 399/805] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:34:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:14.296896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:14.307214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6abc0d-b61f-4170-b20c-cfb712b3fc11 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.329160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:14.336598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:14.348831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.350440 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.350440 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.350440 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.350782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:14.352175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:14.361685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:14.362749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:14.363603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 407/806] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 8 13:34:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:14.364410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:14.409158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576e5d0-db7e-4ab0-9843-867d9d2334f6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.446756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7ae2-b25e-4c82-b2fb-a84720b85153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.447411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7ae2-b25e-4c82-b2fb-a84720b85153 None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.447690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7ae2-b25e-4c82-b2fb-a84720b85153 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.447967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7ae2-b25e-4c82-b2fb-a84720b85153 None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/136ce464d3194f98952beed2d89b8bc3/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.449919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f7ae2-b25e-4c82-b2fb-a84720b85153 None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=136ce464d3194f98952beed2d89b8bc3, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:14.451373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f7ae2-b25e-4c82-b2fb-a84720b85153 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:14.470373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fcb89-f502-4e34-a88b-3297fe7dc538 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.503569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.504169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.504487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.504765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.505426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:14.505730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:14.507166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:14.509077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 400/807] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:34:14 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/136ce464d3194f98952beed2d89b8bc3/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:34:14.509902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7ae2-b25e-4c82-b2fb-a84720b85153 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:14.515944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 408/808] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:34:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:14.516818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:14.577860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:14.628546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:14.639780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79b0c-0229-405b-a5f3-03d7564e37c0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.645578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7ae2-b25e-4c82-b2fb-a84720b85153 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.690460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02da9e-1ecc-400b-a7a0-d07d8dcebd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.691012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02da9e-1ecc-400b-a7a0-d07d8dcebd85 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.691295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02da9e-1ecc-400b-a7a0-d07d8dcebd85 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.691813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02da9e-1ecc-400b-a7a0-d07d8dcebd85 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.692057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91decbc-1085-4e2f-8a9e-0a47d678d396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.692247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02da9e-1ecc-400b-a7a0-d07d8dcebd85 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:14.692509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02da9e-1ecc-400b-a7a0-d07d8dcebd85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:14.692723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91decbc-1085-4e2f-8a9e-0a47d678d396 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.693008 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91decbc-1085-4e2f-8a9e-0a47d678d396 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.693278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91decbc-1085-4e2f-8a9e-0a47d678d396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.693882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02da9e-1ecc-400b-a7a0-d07d8dcebd85 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:14.699081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 409/809] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 13:34:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:14.700285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56687091-1580-41e2-a529-503386a66be0 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.700569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56687091-1580-41e2-a529-503386a66be0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.700840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56687091-1580-41e2-a529-503386a66be0 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.702186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 410/810] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:14.703221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56687091-1580-41e2-a529-503386a66be0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:14.768120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91decbc-1085-4e2f-8a9e-0a47d678d396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:14.769826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91decbc-1085-4e2f-8a9e-0a47d678d396 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:14.811144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56687091-1580-41e2-a529-503386a66be0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:14.825589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:14.846163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:14.846817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.847097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.847368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.858783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:14.860241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:14.867443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:14.875293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:14.879705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:14.881221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:14.881221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:14.881221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:14.881221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 411/811] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:14.881221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:14.889198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 401/812] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:14.889198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1454906-a948-4f4e-80be-4fbed4c4648f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:14.889198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1454906-a948-4f4e-80be-4fbed4c4648f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:14.889198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1454906-a948-4f4e-80be-4fbed4c4648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:14.904770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c1454906-a948-4f4e-80be-4fbed4c4648f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:14.905989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c1454906-a948-4f4e-80be-4fbed4c4648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:14.986724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 402/813] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:14.986724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1454906-a948-4f4e-80be-4fbed4c4648f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.019492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa3ac7-d8fe-4d4b-9a24-3d234eeba331 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.056818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.057557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.058235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.059418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.060135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:15.060511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.061964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.072455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 412/814] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 13:34:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:15.074125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.106794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1454906-a948-4f4e-80be-4fbed4c4648f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.109683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:15.131768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:15.152955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.153656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.154190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.154476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.155189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:15.155494 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.157531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.170264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 403/815] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:15.171207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.190988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787107d-b738-4f62-9a34-9031e4c5e7ed None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.229364 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.230452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.230987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.231459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.246523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.246523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:15.270093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:15.276284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:15.278420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:15.278420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:15.278420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:15.278420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 413/816] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:15.278420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.285753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e3d45-92af-4814-836c-c5028de3cec7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.328236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f9057-6936-483f-91ab-f1e349537010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.328852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f9057-6936-483f-91ab-f1e349537010 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.329127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f9057-6936-483f-91ab-f1e349537010 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.329398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f9057-6936-483f-91ab-f1e349537010 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.330062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f9057-6936-483f-91ab-f1e349537010 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.331441 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f9057-6936-483f-91ab-f1e349537010 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.368297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:15.384739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b981478-9d73-4795-82ae-7fadee1d5826 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.417928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.418583 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.418858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.419124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.426290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.427711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:15.438996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:15.440044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 414/817] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:34:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:15.440983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.462589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 404/818] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:34:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:15.462589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f9057-6936-483f-91ab-f1e349537010 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.510118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:15.572215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92812b-1eee-4f1c-8446-07af7614eda4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.612430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f9057-6936-483f-91ab-f1e349537010 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.617364 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.618000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.618310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.618583 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] PATH_INFO: `/v3/services/network` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.619173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.620603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:15.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:15.632406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:34:15.633446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:15.639718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 415/819] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:15 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:15.640693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.651024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:15.663083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.668846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.668846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.668846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.668846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:15.668846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.668846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.681708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 405/820] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:15.682837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.767498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacc97-9c30-49b1-86f7-0fe7a258040a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.806515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:15.809855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.810827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.811114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.811442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.812629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:15.812946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.814894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.818275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070d13a-ddd3-4887-9ef6-769e29544766 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.843437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 416/821] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 13:34:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:15.843437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.860864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.861884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.862426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.862906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.863747 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] RBAC: Adding query filter params (name=network) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:15.864272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.865960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:15.871322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 406/822] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:15.872454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:15.910710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:15.950071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5deb9d-247a-4f6f-9c60-60f6d7b5b13a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.980934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3297-7fc6-4ea5-95aa-d5159f4ffd34 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:15.986972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:15.987612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:15.987892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:15.988170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:15.996725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:15.998915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:16.014487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:16.015449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:16.016400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 417/823] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:16.016925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:16.021654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:16.022606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.023104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.023583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.034663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:16.036655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:16.052757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:16.053740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:16.054695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:16.054695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:16.054695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:16.061666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 407/824] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:16.062843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:16.075045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:16.140710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9297955-8225-4aa2-bb5d-134d08a5afde None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:16.171663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a42dc-01b8-41e7-826b-a7273af00375 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:16.174513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:16.189323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:16.189323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.189323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.189323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.189714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] RBAC: Adding query filter params (type=network) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:16.189714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:16.189714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.194266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 418/825] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:16.195083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:16.226337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:16.226948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.227232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.227503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.228199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:16.228501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:16.230004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.248734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 408/826] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:16.249645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:16.322846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:16.339629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b11b4-970e-4300-87c7-714b0ba9f297 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:16.371720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9196e-e554-4422-abc1-aca90ea57503 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:16.376611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:16.377221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.377498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.377770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.378442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:16.378806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:16.380231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.392732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 419/827] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:16.393625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:16.418987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966a130-0117-4c3e-80d4-49948a1b4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:16.420309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966a130-0117-4c3e-80d4-49948a1b4e0c None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.421010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966a130-0117-4c3e-80d4-49948a1b4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.421432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966a130-0117-4c3e-80d4-49948a1b4e0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.422935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966a130-0117-4c3e-80d4-49948a1b4e0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:16.424748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966a130-0117-4c3e-80d4-49948a1b4e0c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.447270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:16.500434 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e728a-94ab-4a8a-8089-0c429dc35d02 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:16.518045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 409/828] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:34:16 2022] POST /identity/v3/endpoints => generated 337 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:16.519820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966a130-0117-4c3e-80d4-49948a1b4e0c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:16.564864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:16.565579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.565861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.566131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.566911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=c24d70e571c64f018aa199c4c08d153e) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:16.567225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:16.569658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.587213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 420/829] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:16 2022] GET /identity/v3/role_assignments?user.id=c24d70e571c64f018aa199c4c08d153e&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=97f11873550e41dba0cd095afb1dec00 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:16.685691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966a130-0117-4c3e-80d4-49948a1b4e0c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:16.713654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:16.727235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b086574-cbe9-416a-bc78-fe3fcf354150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:16.728704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b086574-cbe9-416a-bc78-fe3fcf354150 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.728704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b086574-cbe9-416a-bc78-fe3fcf354150 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.728704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b086574-cbe9-416a-bc78-fe3fcf354150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.737248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b086574-cbe9-416a-bc78-fe3fcf354150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:16.738821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b086574-cbe9-416a-bc78-fe3fcf354150 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-0b086574-cbe9-416a-bc78-fe3fcf354150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:16.752892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:16.753918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:16.754855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 410/830] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:16.763389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:16.825154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3226908-8421-4b1b-8f6b-a80a0eca5876 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.825485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3226908-8421-4b1b-8f6b-a80a0eca5876 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.825698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3226908-8421-4b1b-8f6b-a80a0eca5876 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.827606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 411/831] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:16.839798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1915e-9d99-454e-903e-8c971aac2c67 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.840406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1915e-9d99-454e-903e-8c971aac2c67 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.840889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1915e-9d99-454e-903e-8c971aac2c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.862625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-cab1915e-9d99-454e-903e-8c971aac2c67 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:16.863727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cab1915e-9d99-454e-903e-8c971aac2c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:16.906865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34169bbe-2279-4283-b772-0dde92b2fad9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:16.920075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:16.960041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 412/832] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:16.968642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:16.969320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:16.969661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:16.969945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:16.970729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:16.971039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:16.972492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:16.985471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 421/833] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:16.991203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1915e-9d99-454e-903e-8c971aac2c67 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:17.099093 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1915e-9d99-454e-903e-8c971aac2c67 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:17.114167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:17.135124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd6a11-4791-43e8-a218-a553977cd74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:17.135844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd6a11-4791-43e8-a218-a553977cd74a None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:17.136123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd6a11-4791-43e8-a218-a553977cd74a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:17.136396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd6a11-4791-43e8-a218-a553977cd74a None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/df727be6307948c49affa47e601f989c/roles/04bab2a5f02f4140b88885a57b278b5d` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:17.151887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd6a11-4791-43e8-a218-a553977cd74a None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=df727be6307948c49affa47e601f989c, role_id=04bab2a5f02f4140b88885a57b278b5d)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:17.153336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd6a11-4791-43e8-a218-a553977cd74a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:17.227788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 413/834] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:34:16 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/df727be6307948c49affa47e601f989c/roles/04bab2a5f02f4140b88885a57b278b5d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:34:17.243627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:17.368366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f3b2c-40f8-4525-95cd-7b644f6dcda9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:17.405244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af41ac-4273-43ab-9264-c1f616bd6852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:17.405934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af41ac-4273-43ab-9264-c1f616bd6852 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:17.406286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af41ac-4273-43ab-9264-c1f616bd6852 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:17.406568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af41ac-4273-43ab-9264-c1f616bd6852 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:17.418884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af41ac-4273-43ab-9264-c1f616bd6852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-41af41ac-4273-43ab-9264-c1f616bd6852 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:17.424446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:17.425448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:17.426318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:17.430305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 422/835] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:17.432199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd6a11-4791-43e8-a218-a553977cd74a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:17.536851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd6a11-4791-43e8-a218-a553977cd74a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:17.574069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:17.574706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:17.574987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:17.575259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:17.575935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:17.576235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:17.577603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:17.586819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 414/836] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:17.592749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af41ac-4273-43ab-9264-c1f616bd6852 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:17.622298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:17.697210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af41ac-4273-43ab-9264-c1f616bd6852 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:17.734936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86705a5-db83-4c62-817a-4350c18d4000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:17.735594 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86705a5-db83-4c62-817a-4350c18d4000 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:17.735874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86705a5-db83-4c62-817a-4350c18d4000 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:17.736144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86705a5-db83-4c62-817a-4350c18d4000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:17.749608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86705a5-db83-4c62-817a-4350c18d4000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:17.751151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86705a5-db83-4c62-817a-4350c18d4000 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-e86705a5-db83-4c62-817a-4350c18d4000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:17.767460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:17.768489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:17.769410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:17.769410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:17.769410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:17.769410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 423/837] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:17.775262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:17.806767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:17.848166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a1a2-a33a-4093-8a72-fe1c59e141d4 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:17.848495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a1a2-a33a-4093-8a72-fe1c59e141d4 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:17.848713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a1a2-a33a-4093-8a72-fe1c59e141d4 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:17.850337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 424/838] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:17.855396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8f06e-7a8d-4c87-ab03-4178acc3320b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:17.855773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8f06e-7a8d-4c87-ab03-4178acc3320b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:17.856052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8f06e-7a8d-4c87-ab03-4178acc3320b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:17.878808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c4d8f06e-7a8d-4c87-ab03-4178acc3320b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:17.879937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c4d8f06e-7a8d-4c87-ab03-4178acc3320b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:17.892570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b98e21-f597-401f-b8e0-552b7a50a377 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:17.929570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:17.930357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:17.930642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:17.930921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:17.931618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:17.931923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:17.934061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:17.951262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 425/840] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:17.951834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 415/840] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:17.955171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8f06e-7a8d-4c87-ab03-4178acc3320b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:17.972072 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:18.083291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8f06e-7a8d-4c87-ab03-4178acc3320b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:18.121486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:18.122130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:18.122429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:18.122702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:18.130347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:18.131738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:18.145348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:18.147545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:18.148793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 426/841] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:18.153132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:18.228663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:18.253282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:18.272807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27227e-a73e-419b-a1da-d11b5264ad3c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:18.308772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:18.309429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:18.310047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:18.310356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:18.310630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:18.311356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:18.311658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:18.313125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:18.332712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 416/842] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 13:34:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:18.339277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:18.347179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f599ab-86bb-445c-bad7-d8bd1133ad33 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:18.393312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:18.393873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:18.394178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:18.394453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:18.395120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:18.396526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:18.407961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:18.410328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:18.411455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 427/843] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:18.415800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:18.459331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf6e85-df2b-4375-89a7-08c797c0ea9c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:18.501116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bb5c7-189c-4219-ac05-45476136e651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:18.501726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bb5c7-189c-4219-ac05-45476136e651 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:18.502003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bb5c7-189c-4219-ac05-45476136e651 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:18.502302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bb5c7-189c-4219-ac05-45476136e651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:18.517809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:18.533327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb46c31-df0d-4150-b2db-22271aafaec3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:18.572622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:18.573400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:18.573780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:18.574165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:18.574967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:18.575372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:18.577207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:18.578183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:18.579462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bb5c7-189c-4219-ac05-45476136e651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:18.581079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bb5c7-189c-4219-ac05-45476136e651 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:18.587297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 428/844] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:18.589032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:18.714509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea509-9219-453c-a0f4-9dfb0133fefc None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:18.716754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 417/845] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:18.719642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bb5c7-189c-4219-ac05-45476136e651 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:18.756899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:18.758201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:18.759131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:18.759893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:18.762306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:18.763146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:18.766020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:18.778294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 429/846] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:18.781738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:18.855915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bb5c7-189c-4219-ac05-45476136e651 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:18.894696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234bfa8-1759-43cf-85a6-155fb78d3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:18.895347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234bfa8-1759-43cf-85a6-155fb78d3e27 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:18.895678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234bfa8-1759-43cf-85a6-155fb78d3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:18.895962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234bfa8-1759-43cf-85a6-155fb78d3e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:18.908340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234bfa8-1759-43cf-85a6-155fb78d3e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:18.909817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234bfa8-1759-43cf-85a6-155fb78d3e27 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-a234bfa8-1759-43cf-85a6-155fb78d3e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:18.927206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:18.928222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:18.929142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:18.929142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:18.929142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:18.932273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 418/847] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:18.935293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:19.011388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:19.052152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e188eaa-d622-41e7-9e90-0788f213991e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:19.089404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:19.090019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.090334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.090607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.091255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:19.091820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:19.093191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:19.100401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485ee3-baeb-4a42-8e3e-e41eca3c250c None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.100686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485ee3-baeb-4a42-8e3e-e41eca3c250c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.100953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485ee3-baeb-4a42-8e3e-e41eca3c250c None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.102655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 419/848] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:19.107823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 430/849] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:19.114100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b6836-6147-435c-b530-d9392c3df01b None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.114498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b6836-6147-435c-b530-d9392c3df01b None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.114844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b6836-6147-435c-b530-d9392c3df01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.119156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:19.134914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-681b6836-6147-435c-b530-d9392c3df01b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:19.137422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-681b6836-6147-435c-b530-d9392c3df01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:19.230227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 420/850] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:19.253649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d1a59-13af-43fd-aed3-f80d34762c9f None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:19.280280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:19.294409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:19.295051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.295391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.295661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.303634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:19.305348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:19.318405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:19.319333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:19.320273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 431/851] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:19.326314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b6836-6147-435c-b530-d9392c3df01b None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:19.356863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:19.433414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b6836-6147-435c-b530-d9392c3df01b None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:19.468960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:19.469581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.469865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.470160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.470862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:19.471170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:19.472658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:19.490252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 421/852] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:19.491691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:19.522226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:19.626220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a535444-938d-4c6c-a23f-5a2d926ee2a9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:19.641778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:19.660694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87935a4-73ec-446f-8f30-9894cbe157ee None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:19.663554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb09bb-8e65-4102-9a34-d838f29e06fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:19.664200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb09bb-8e65-4102-9a34-d838f29e06fb None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.664476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb09bb-8e65-4102-9a34-d838f29e06fb None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.664746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb09bb-8e65-4102-9a34-d838f29e06fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.665488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb09bb-8e65-4102-9a34-d838f29e06fb None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=136ce464d3194f98952beed2d89b8bc3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:19.665799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb09bb-8e65-4102-9a34-d838f29e06fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:19.667927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb09bb-8e65-4102-9a34-d838f29e06fb None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:19.674781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 432/853] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:19 2022] GET /identity/v3/role_assignments?user.id=136ce464d3194f98952beed2d89b8bc3&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=97f11873550e41dba0cd095afb1dec00 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:19.676037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822a077-1ed8-45b5-b8b8-21d56a50189f None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.676318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822a077-1ed8-45b5-b8b8-21d56a50189f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.676587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822a077-1ed8-45b5-b8b8-21d56a50189f None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.682089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 433/854] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:19.684301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea1062-6e33-4817-ba06-6a5928c773dc None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.684582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea1062-6e33-4817-ba06-6a5928c773dc None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.684852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea1062-6e33-4817-ba06-6a5928c773dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.709161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1cea1062-6e33-4817-ba06-6a5928c773dc None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:19.711380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1cea1062-6e33-4817-ba06-6a5928c773dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:19.718739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:19.719397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.719676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.719946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.720547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:19.721997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:19.734503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:19.735540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:19.736445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:19.741287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 422/855] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 8 13:34:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:19.742811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:19.781378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:19.831681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 434/856] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:19.894728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03cef3-9d95-433b-a9cc-56bc7db9ed89 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:19.931659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:19.932902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:19.933192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:19.933475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:19.934124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:19.934457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:19.935920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:19.941501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 423/857] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:19.947710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea1062-6e33-4817-ba06-6a5928c773dc None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:20.063707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea1062-6e33-4817-ba06-6a5928c773dc None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:20.104934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:20.105557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:20.105835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:20.106108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:20.106779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:20.107081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:20.108455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:20.118554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 435/858] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:20.135305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:20.155967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:20.259857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd230d2-69ab-4aed-95a0-baa4b4293e60 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:20.303310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516a452-107d-4889-9ab2-0174c28ad448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:20.304545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516a452-107d-4889-9ab2-0174c28ad448 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:20.304991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516a452-107d-4889-9ab2-0174c28ad448 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:20.305270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516a452-107d-4889-9ab2-0174c28ad448 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:20.323125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516a452-107d-4889-9ab2-0174c28ad448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-9516a452-107d-4889-9ab2-0174c28ad448 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:20.328473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:20.329927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:20.330947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 424/859] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:34:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:20.338616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:20.386230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:20.459694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361e704-e57c-43a9-96bf-fa6812452728 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:20.499529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:20.500552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:20.501040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:20.501524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:20.502424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:20.502938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:20.504980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:20.514783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 436/860] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:34:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:20.520883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516a452-107d-4889-9ab2-0174c28ad448 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:20.611523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:20.641089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516a452-107d-4889-9ab2-0174c28ad448 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:20.676789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:20.677429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:20.677714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:20.677991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:20.690625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:20.692096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:20.708324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:20.709398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:20.710389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:20.710389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:20.710389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:20.710389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 425/861] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:20.718403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:20.826345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58da2f5-3ebd-4919-9e01-6412ab5265b1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:20.828243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:20.861826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:20.862987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:20.863481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:20.863953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:20.864882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:20.865388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:20.866984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:20.880441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 437/862] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:20.886793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:20.996081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07662-a4ac-43f0-835f-9863a0b6ff17 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:21.028861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:21.037805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dacdd-9e8a-4844-9b91-8e6da9b9e03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:21.038604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dacdd-9e8a-4844-9b91-8e6da9b9e03b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:21.038889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dacdd-9e8a-4844-9b91-8e6da9b9e03b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:21.039166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dacdd-9e8a-4844-9b91-8e6da9b9e03b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:21.047354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dacdd-9e8a-4844-9b91-8e6da9b9e03b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:21.049149 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dacdd-9e8a-4844-9b91-8e6da9b9e03b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-731dacdd-9e8a-4844-9b91-8e6da9b9e03b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:21.061796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:21.062892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:21.063860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 426/863] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:34:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:21.073332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:21.102268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02534ca0-be38-45d6-95fe-5cffaf418284 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:21.102539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02534ca0-be38-45d6-95fe-5cffaf418284 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:21.102773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02534ca0-be38-45d6-95fe-5cffaf418284 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:21.104762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 427/864] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:21.111212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c398c-c13c-4755-bcb6-cb839709de5d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:21.111497 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c398c-c13c-4755-bcb6-cb839709de5d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:21.111778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c398c-c13c-4755-bcb6-cb839709de5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:21.130715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2b8c398c-c13c-4755-bcb6-cb839709de5d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:21.131918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2b8c398c-c13c-4755-bcb6-cb839709de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:21.187789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819cf34-6206-4fd0-b22d-9cf74adb0e41 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:21.202217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 428/865] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:21.233169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:21.234218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:21.234729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:21.235267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:21.236318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:21.236863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:21.238581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:21.260598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 438/866] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 13:34:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:21.263360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c398c-c13c-4755-bcb6-cb839709de5d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:21.372048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c398c-c13c-4755-bcb6-cb839709de5d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:21.389096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:21.407674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb65e2-d0d4-48e0-b3e1-4a6254ba73dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:21.408397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb65e2-d0d4-48e0-b3e1-4a6254ba73dd None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:21.408673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb65e2-d0d4-48e0-b3e1-4a6254ba73dd None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:21.408940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb65e2-d0d4-48e0-b3e1-4a6254ba73dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:21.481307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb65e2-d0d4-48e0-b3e1-4a6254ba73dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:21.482857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb65e2-d0d4-48e0-b3e1-4a6254ba73dd None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:21.547329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:21.569673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:21.614174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 429/867] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:21 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:21.615085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb65e2-d0d4-48e0-b3e1-4a6254ba73dd None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:21.641561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:21.686818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394248b-f947-4806-988f-567b5644f1c2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:21.730113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313105-5b15-40ce-b92c-e163de74dc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:21.730865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313105-5b15-40ce-b92c-e163de74dc37 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:21.731228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313105-5b15-40ce-b92c-e163de74dc37 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:21.731645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313105-5b15-40ce-b92c-e163de74dc37 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:21.732345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85313105-5b15-40ce-b92c-e163de74dc37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:21.733862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85313105-5b15-40ce-b92c-e163de74dc37 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:21.743393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb65e2-d0d4-48e0-b3e1-4a6254ba73dd None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:21.780973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:21.781663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:21.781949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:21.782327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:21.793719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:21.795324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:21.803336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 439/868] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:34:21 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:21.822022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:21.823645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:21.824537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:21.824537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:21.824537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:21.824537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 430/869] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:21.830589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313105-5b15-40ce-b92c-e163de74dc37 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:21.925889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:21.954961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313105-5b15-40ce-b92c-e163de74dc37 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:21.999504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad6a37-3b5c-4d42-bf1b-884c3d0a9959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:22.000130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad6a37-3b5c-4d42-bf1b-884c3d0a9959 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.000405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad6a37-3b5c-4d42-bf1b-884c3d0a9959 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.000672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad6a37-3b5c-4d42-bf1b-884c3d0a9959 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.002191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad6a37-3b5c-4d42-bf1b-884c3d0a9959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:22.002191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad6a37-3b5c-4d42-bf1b-884c3d0a9959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:22.003017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad6a37-3b5c-4d42-bf1b-884c3d0a9959 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:22.016557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 440/870] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:22.021989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:22.089679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159fb44-9d44-4e69-aa57-6e93397aa30a None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.090679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159fb44-9d44-4e69-aa57-6e93397aa30a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.091029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159fb44-9d44-4e69-aa57-6e93397aa30a None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.092860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 441/871] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:22.099582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182958dc-e0e4-4fe4-847b-d6b97f6f1b38 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.099871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182958dc-e0e4-4fe4-847b-d6b97f6f1b38 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.100152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182958dc-e0e4-4fe4-847b-d6b97f6f1b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.104813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:22.119182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-182958dc-e0e4-4fe4-847b-d6b97f6f1b38 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:22.126832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-182958dc-e0e4-4fe4-847b-d6b97f6f1b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:22.149218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e63907-e20c-4bd8-9d39-3640e07d1008 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:22.196046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f53d50-c45b-4047-873f-176b32a069f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:22.201133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 442/872] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:22.201275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f53d50-c45b-4047-873f-176b32a069f2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.201275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f53d50-c45b-4047-873f-176b32a069f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.201275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f53d50-c45b-4047-873f-176b32a069f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.210709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f53d50-c45b-4047-873f-176b32a069f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:22.212240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f53d50-c45b-4047-873f-176b32a069f2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-45f53d50-c45b-4047-873f-176b32a069f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:22.224946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:22.225871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:22.226871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 431/873] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:22.232069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182958dc-e0e4-4fe4-847b-d6b97f6f1b38 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:22.308412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:22.344489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182958dc-e0e4-4fe4-847b-d6b97f6f1b38 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:22.381878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:22.382879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.383429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.383918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.385555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:22.386078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:22.387723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:22.397400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34a129-b532-4784-bf18-4f54c0fda0cd None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.397894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34a129-b532-4784-bf18-4f54c0fda0cd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.397988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34a129-b532-4784-bf18-4f54c0fda0cd None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.399462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 432/874] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:22.401695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 443/875] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:22.407572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:22.407938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdcc62-8090-4f3a-be01-795ed80e83c6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.408283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdcc62-8090-4f3a-be01-795ed80e83c6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.408597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdcc62-8090-4f3a-be01-795ed80e83c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.429762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2dfdcc62-8090-4f3a-be01-795ed80e83c6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:22.430990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2dfdcc62-8090-4f3a-be01-795ed80e83c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:22.504711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 433/876] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:22.506785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdcc62-8090-4f3a-be01-795ed80e83c6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:22.533287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ebdd-a1cd-4b09-9653-ac5e6c4b6dbc None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:22.559292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:22.568027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:22.568831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.569253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.569689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.570583 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=df727be6307948c49affa47e601f989c) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:22.571033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:22.573169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:22.584009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:22.584144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 444/877] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:22 2022] GET /identity/v3/role_assignments?user.id=df727be6307948c49affa47e601f989c&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:22.644554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdcc62-8090-4f3a-be01-795ed80e83c6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:22.685157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:22.685819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.686104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.686410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.687014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:22.688406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:22.698450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:22.699557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:22.700413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 434/878] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:22.705984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:22.789235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:22.825293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53891a89-74d2-4114-bdac-65e308449a64 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:22.863626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:22.864236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.864519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.864796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.865430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:22.865733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:22.867140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:22.875592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 445/879] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:22.880691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:22.917895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941a3f-04a6-4cf9-9d4e-79c53cb5761e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:22.944084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:22.953885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:22.960181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171fea5-aafe-446d-ada6-d7f16d76581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:22.961666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171fea5-aafe-446d-ada6-d7f16d76581d None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:22.961666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171fea5-aafe-446d-ada6-d7f16d76581d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:22.961666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171fea5-aafe-446d-ada6-d7f16d76581d None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:22.962052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171fea5-aafe-446d-ada6-d7f16d76581d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:22.963488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171fea5-aafe-446d-ada6-d7f16d76581d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:23.009446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3e7e3-5bfc-4c05-835b-7c9b67fbaef0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:23.043909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e269f-b3fd-488a-a6b1-24a105905359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:23.044528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e269f-b3fd-488a-a6b1-24a105905359 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:23.044812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e269f-b3fd-488a-a6b1-24a105905359 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:23.045158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e269f-b3fd-488a-a6b1-24a105905359 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:23.045794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e269f-b3fd-488a-a6b1-24a105905359 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:23.046101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e269f-b3fd-488a-a6b1-24a105905359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:23.047626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e269f-b3fd-488a-a6b1-24a105905359 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:23.053582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 435/880] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:34:22 2022] POST /identity/v3/services => generated 269 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:23.058157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 446/881] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:24.111290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b527b4b-b286-41b3-bc33-80c1fe758860 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:24.111867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b527b4b-b286-41b3-bc33-80c1fe758860 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:24.111867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b527b4b-b286-41b3-bc33-80c1fe758860 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:24.113385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 436/882] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:24.118744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ce4a8-17ba-4fb4-8bcc-8b6b44286b50 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:24.119186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ce4a8-17ba-4fb4-8bcc-8b6b44286b50 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:24.119393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ce4a8-17ba-4fb4-8bcc-8b6b44286b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:24.138268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a02ce4a8-17ba-4fb4-8bcc-8b6b44286b50 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:24.139447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a02ce4a8-17ba-4fb4-8bcc-8b6b44286b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:24.216770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 447/883] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:24.485895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b527b4b-b286-41b3-bc33-80c1fe758860 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:24.595145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b527b4b-b286-41b3-bc33-80c1fe758860 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:24.629907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:24.630565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:24.630846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:24.631119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:24.631702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:24.638458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:24.647152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:24.648146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:24.648990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:24.653945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 437/884] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:24.654921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ce4a8-17ba-4fb4-8bcc-8b6b44286b50 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:24.702867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:24.763763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ce4a8-17ba-4fb4-8bcc-8b6b44286b50 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:24.799881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93086603-8991-4cf7-bed7-50d0602b4c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:24.800496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93086603-8991-4cf7-bed7-50d0602b4c66 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:24.800771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93086603-8991-4cf7-bed7-50d0602b4c66 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:24.801040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93086603-8991-4cf7-bed7-50d0602b4c66 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:24.801667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93086603-8991-4cf7-bed7-50d0602b4c66 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:24.801969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93086603-8991-4cf7-bed7-50d0602b4c66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:24.803476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93086603-8991-4cf7-bed7-50d0602b4c66 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:24.809291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 448/885] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:24.815101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:24.870522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:24.924333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21c3a-ad9c-4d38-b0ae-61dfced19ff2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:24.954782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1ba88-82cf-4584-ace3-c049af7d928c None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:24.955116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1ba88-82cf-4584-ace3-c049af7d928c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:24.955329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1ba88-82cf-4584-ace3-c049af7d928c None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:24.956919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 449/886] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:24.960502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:24.961363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:24.961872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:24.962037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d0ed-e85e-43cb-8b9d-fcf6eede09e6 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:24.962374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d0ed-e85e-43cb-8b9d-fcf6eede09e6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:24.962684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d0ed-e85e-43cb-8b9d-fcf6eede09e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:24.963129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:24.963974 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:24.964478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:24.966694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:24.974018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 438/887] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:24.981091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:24.985963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-83e2d0ed-e85e-43cb-8b9d-fcf6eede09e6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:24.987014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-83e2d0ed-e85e-43cb-8b9d-fcf6eede09e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:25.071092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 450/888] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:25.099488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9148f-7fb9-4d13-ab8a-4faeab14caa2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:25.111002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:25.135746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f0a0a-dcff-488f-b617-83c239ec6ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:25.137073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f0a0a-dcff-488f-b617-83c239ec6ea0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.137552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f0a0a-dcff-488f-b617-83c239ec6ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.137996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f0a0a-dcff-488f-b617-83c239ec6ea0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.138940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f0a0a-dcff-488f-b617-83c239ec6ea0 None admin] RBAC: Adding query filter params (interface=public, service_id=49aa5cd2775c4c11a32f7c0c2e987647, region_id=RegionOne) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:25.139416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f0a0a-dcff-488f-b617-83c239ec6ea0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:25.140984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f0a0a-dcff-488f-b617-83c239ec6ea0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:25.146838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 439/889] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 13:34:24 2022] GET /identity/v3/endpoints?service_id=49aa5cd2775c4c11a32f7c0c2e987647&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:25.155290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d0ed-e85e-43cb-8b9d-fcf6eede09e6 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:25.199263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:25.269410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d0ed-e85e-43cb-8b9d-fcf6eede09e6 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:25.309732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4cc3-fb19-448a-ac53-9262c5ea4c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:25.310820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4cc3-fb19-448a-ac53-9262c5ea4c05 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.311328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4cc3-fb19-448a-ac53-9262c5ea4c05 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.311829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4cc3-fb19-448a-ac53-9262c5ea4c05 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.312674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4cc3-fb19-448a-ac53-9262c5ea4c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:25.314388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4cc3-fb19-448a-ac53-9262c5ea4c05 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:25.321593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 451/890] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:34:25 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:34:25.340790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a14a1c-7bad-41f5-8699-f8042041da23 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.341330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a14a1c-7bad-41f5-8699-f8042041da23 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.341754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a14a1c-7bad-41f5-8699-f8042041da23 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.343277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 440/891] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:25.348973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354aab1-dc03-4ea5-998d-cc02812b1dfd None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.349304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354aab1-dc03-4ea5-998d-cc02812b1dfd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.349529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354aab1-dc03-4ea5-998d-cc02812b1dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.370249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-7354aab1-dc03-4ea5-998d-cc02812b1dfd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:25.371489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7354aab1-dc03-4ea5-998d-cc02812b1dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:25.397227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a14a1c-7bad-41f5-8699-f8042041da23 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:25.455801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 452/892] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:25.457095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ed886-785a-4737-b405-cf5869c1155a None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.457380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ed886-785a-4737-b405-cf5869c1155a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.457650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ed886-785a-4737-b405-cf5869c1155a None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.464007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 453/893] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:25.465425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda5ee9-11d8-4ef6-abce-64a176179848 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.465703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda5ee9-11d8-4ef6-abce-64a176179848 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.466277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda5ee9-11d8-4ef6-abce-64a176179848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.489392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-dcda5ee9-11d8-4ef6-abce-64a176179848 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:25.490660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-dcda5ee9-11d8-4ef6-abce-64a176179848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:25.522793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:25.531739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a14a1c-7bad-41f5-8699-f8042041da23 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:25.577014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44c04a-8dbe-4c0a-960f-db32afc4a0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:25.577574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 454/894] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:25.578231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44c04a-8dbe-4c0a-960f-db32afc4a0b7 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.578671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44c04a-8dbe-4c0a-960f-db32afc4a0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.579080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44c04a-8dbe-4c0a-960f-db32afc4a0b7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.591877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44c04a-8dbe-4c0a-960f-db32afc4a0b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-2d44c04a-8dbe-4c0a-960f-db32afc4a0b7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:25.597116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:25.598051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:25.598935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:25.601252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 441/895] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 13:34:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:25.607610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda5ee9-11d8-4ef6-abce-64a176179848 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:25.734023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda5ee9-11d8-4ef6-abce-64a176179848 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:25.746586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44c04a-8dbe-4c0a-960f-db32afc4a0b7 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:25.774945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:25.776582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.777335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.778084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.779115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:25.779606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:25.781674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:25.782946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:25.792858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 455/896] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 13:34:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:25.798706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:25.831288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:25.855423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44c04a-8dbe-4c0a-960f-db32afc4a0b7 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:25.896674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32752cc5-522b-47e2-9518-351fe1a368e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:25.897283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32752cc5-522b-47e2-9518-351fe1a368e4 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.897560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32752cc5-522b-47e2-9518-351fe1a368e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.897830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32752cc5-522b-47e2-9518-351fe1a368e4 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.898714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32752cc5-522b-47e2-9518-351fe1a368e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:25.900121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32752cc5-522b-47e2-9518-351fe1a368e4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:25.920495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a3ef3-d8fd-4954-aa12-2b4277b6f8d8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:25.954504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:25.955139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:25.955434 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:25.956267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:25.966544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:25.968108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:25.981608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 442/897] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:34:25 2022] POST /identity/v3/services => generated 241 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:34:25.982648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32752cc5-522b-47e2-9518-351fe1a368e4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:25.982772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:25.983692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:25.984637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:25.984637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:25.984637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:25.984637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 456/898] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:25.992205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.089719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:26.115567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64347117-0226-4684-a16b-8c36c078e857 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.116750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:26.137021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32752cc5-522b-47e2-9518-351fe1a368e4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.158204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.159488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.159945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.160404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.161251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:26.161872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.163495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:26.177349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 457/899] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:26.183439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.188331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.189314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.193956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.194408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.195241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.196775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:26.207172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:26.208326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:26.209415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:26.212128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 443/900] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 8 13:34:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:26.218625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.329648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23406c-3a0c-4d73-9483-58a9cf3d8c7c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.343662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cfc3b-b201-47cb-8b73-4dee6550f68d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.366389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.367317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.367774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.368197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.372385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:26.377809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.379485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:26.389117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.389736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.390016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.390327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.394602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:26.394708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:26.395580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:26.396540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:26.397074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.397144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 458/901] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:34:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:26.398830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:26.399922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.405718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 444/902] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:26.411872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.442044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:26.516440 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641e8b2-f25a-407c-8836-6ce15774cb62 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.555771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829624d4-c343-45ba-9f99-9d11b6bcf600 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.557867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.558755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.559248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.559715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.560658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:26.561159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.562820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:26.581460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 459/903] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 13:34:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:26.587261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.609251 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.609780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.610063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.610370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.611565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:26.611879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.613805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:26.622333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 445/904] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:26.627815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.664477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:26.724661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:26.731609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a7238-311e-4ecd-8df6-f115f25b1a04 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.749709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a826d-a677-426a-81d5-0277c335f011 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.772625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c05619-fe85-43f9-9ba7-646cb13c3ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.773249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c05619-fe85-43f9-9ba7-646cb13c3ec5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.773544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c05619-fe85-43f9-9ba7-646cb13c3ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.773817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c05619-fe85-43f9-9ba7-646cb13c3ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.795633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.796634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.797135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.797614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.798954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] RBAC: Adding query filter params (interface=public, service_id=563208c5429c442e877181f6638b3301, region_id=RegionOne) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:26.799272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.800646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:26.813395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 446/905] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 13:34:26 2022] GET /identity/v3/endpoints?service_id=563208c5429c442e877181f6638b3301&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:26.818842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.859878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c05619-fe85-43f9-9ba7-646cb13c3ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.861664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c05619-fe85-43f9-9ba7-646cb13c3ec5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:26.925818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:26.935069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df8ae8-9021-4c7e-85c9-8bed3b7c963d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:26.983883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 460/906] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:26 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:26.985184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c05619-fe85-43f9-9ba7-646cb13c3ec5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:26.987664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7b9d3-6294-4d9e-9f45-2e46db35e95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:26.989364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7b9d3-6294-4d9e-9f45-2e46db35e95e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:26.989842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7b9d3-6294-4d9e-9f45-2e46db35e95e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:26.990334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7b9d3-6294-4d9e-9f45-2e46db35e95e None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:26.991113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7b9d3-6294-4d9e-9f45-2e46db35e95e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:26.992954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7b9d3-6294-4d9e-9f45-2e46db35e95e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:27.001745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 447/907] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:34:26 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:34:27.126071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c05619-fe85-43f9-9ba7-646cb13c3ec5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:27.163728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:27.165084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:27.166064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:27.166520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:27.183429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:27.185064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:27.204178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:27.205184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:27.206120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:27.206120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:27.206120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:27.206120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 461/908] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:27.212557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7b9d3-6294-4d9e-9f45-2e46db35e95e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:27.227435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:27.326178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7b9d3-6294-4d9e-9f45-2e46db35e95e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:27.362425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1983fc2-5990-4d02-a256-16487a495584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:27.363416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1983fc2-5990-4d02-a256-16487a495584 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:27.363914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1983fc2-5990-4d02-a256-16487a495584 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:27.364385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1983fc2-5990-4d02-a256-16487a495584 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:27.365233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1983fc2-5990-4d02-a256-16487a495584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:27.365737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1983fc2-5990-4d02-a256-16487a495584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:27.367351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1983fc2-5990-4d02-a256-16487a495584 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:27.381354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 448/909] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:27.390218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:27.402267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:27.507069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1322e6f-49c8-4ffe-8807-13e56bf536d2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:27.543316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:27.544301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:27.544824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:27.545298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:27.552452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:27.554382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:27.563521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa610ca-13af-431b-afac-a68fc2384620 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:27.563807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa610ca-13af-431b-afac-a68fc2384620 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:27.564075 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa610ca-13af-431b-afac-a68fc2384620 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:27.565632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 449/910] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:27.568278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:27.569476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:27.570708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:27.570708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:27.570708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:27.570708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:27.570708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:27.570708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:27.570708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:27.570708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:27.571153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8e6f-6af6-4c22-b840-1e5dbcf7b372 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:27.571153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8e6f-6af6-4c22-b840-1e5dbcf7b372 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:27.571153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8e6f-6af6-4c22-b840-1e5dbcf7b372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:27.578297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 462/911] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:27.579697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:27.590162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-437e8e6f-6af6-4c22-b840-1e5dbcf7b372 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:27.591625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-437e8e6f-6af6-4c22-b840-1e5dbcf7b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:27.664189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:27.673067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:27.678867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 450/912] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:27.707228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc4e1f-3092-44d8-9a28-6fc32f6df6ee None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:27.744601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:27.745494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:27.745935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:27.746387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:27.747226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:27.747680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:27.749330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:27.767837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 463/913] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:27.769364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8e6f-6af6-4c22-b840-1e5dbcf7b372 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:27.873744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8e6f-6af6-4c22-b840-1e5dbcf7b372 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:27.909792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:27.910350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:27.910632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:27.910904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:27.911633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] RBAC: Adding query filter params (role.id=1ab14f52bb114cefadb70cd2a83acf19, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=df727be6307948c49affa47e601f989c) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:27.911942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:27.913369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:27.919653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 451/914] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:27 2022] GET /identity/v3/role_assignments?user.id=df727be6307948c49affa47e601f989c&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=1ab14f52bb114cefadb70cd2a83acf19 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:27.943338 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:28.051965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7bfe-8092-4957-a4ae-dd13fd9e2647 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:28.082454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:28.092416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:28.096074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:28.096074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:28.096074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:28.096074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:28.096074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:28.105385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:28.106465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:28.107706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 464/915] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:28.112951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:28.183240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:28.226684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116b27b-9eef-422f-9d5b-c369595aeaba None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:28.267927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc83ad0-9673-450c-a7c8-a0ecd1ad5058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:28.268541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc83ad0-9673-450c-a7c8-a0ecd1ad5058 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:28.268915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc83ad0-9673-450c-a7c8-a0ecd1ad5058 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:28.269194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc83ad0-9673-450c-a7c8-a0ecd1ad5058 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:28.269823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc83ad0-9673-450c-a7c8-a0ecd1ad5058 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:28.270131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc83ad0-9673-450c-a7c8-a0ecd1ad5058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:28.271507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc83ad0-9673-450c-a7c8-a0ecd1ad5058 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:28.276914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 452/916] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:28.282772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:28.313564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3e82-2a21-47d9-b790-329af6123d24 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:28.313876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3e82-2a21-47d9-b790-329af6123d24 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:28.314111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3e82-2a21-47d9-b790-329af6123d24 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:28.315592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 453/917] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:28.320780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198161b-6822-4a3e-946c-a7f945a75f6c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:28.321056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198161b-6822-4a3e-946c-a7f945a75f6c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:28.321348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198161b-6822-4a3e-946c-a7f945a75f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:28.341820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e198161b-6822-4a3e-946c-a7f945a75f6c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:28.343387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e198161b-6822-4a3e-946c-a7f945a75f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:28.407956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0b9ad-e66e-4138-be92-d7872af28602 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:28.422272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 454/918] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:28.448020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:28.448551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:28.448832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:28.449107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:28.449746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:28.450101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:28.451569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:28.457302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 465/919] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 13:34:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:28.462621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198161b-6822-4a3e-946c-a7f945a75f6c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:28.522553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:28.565294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198161b-6822-4a3e-946c-a7f945a75f6c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:28.605295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe0415-323b-418f-b040-436cba569530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:28.605905 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe0415-323b-418f-b040-436cba569530 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:28.606205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe0415-323b-418f-b040-436cba569530 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:28.606480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe0415-323b-418f-b040-436cba569530 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:28.607155 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe0415-323b-418f-b040-436cba569530 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:28.608506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe0415-323b-418f-b040-436cba569530 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:28.687209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 455/920] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:34:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:28.734783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:28.799311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:28.854439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c459e1-1c67-461f-8f9d-fc5b2703f11d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:28.898020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:28.898700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:28.898980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:28.899249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:28.899859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:28.901230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:28.910289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:28.911326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:28.912191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 466/921] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:28.918124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe0415-323b-418f-b040-436cba569530 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:29.054261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe0415-323b-418f-b040-436cba569530 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:29.070545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:29.090647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:29.091643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:29.091930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:29.092258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:29.092962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:29.093268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:29.094669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:29.099916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 456/922] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:29.105964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:29.217703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72932d2b-4d7a-4dfb-b225-134971079b98 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:29.232976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:29.255019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:29.255635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:29.255977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:29.256260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:29.256911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:29.257221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:29.258637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:29.264476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 467/923] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:29.270448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:29.385118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0860483-0e80-46f0-b0b6-7ceff3076980 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:29.420545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ed3a7-7ebf-45f3-bebf-c105fc112744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:29.421166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed3a7-7ebf-45f3-bebf-c105fc112744 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:29.421432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed3a7-7ebf-45f3-bebf-c105fc112744 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:29.421730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed3a7-7ebf-45f3-bebf-c105fc112744 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:29.422412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ed3a7-7ebf-45f3-bebf-c105fc112744 None admin] RBAC: Adding query filter params (interface=public, service_id=2bbc0564ebb94884a14a9f420a5553db, region_id=RegionOne) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:29.422717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ed3a7-7ebf-45f3-bebf-c105fc112744 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:29.424512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ed3a7-7ebf-45f3-bebf-c105fc112744 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:29.430288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 457/924] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 13:34:29 2022] GET /identity/v3/endpoints?service_id=2bbc0564ebb94884a14a9f420a5553db&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:29.435805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:29.458844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3c7d-7cfc-4654-a7d4-652e9b0acaa8 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:29.459158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3c7d-7cfc-4654-a7d4-652e9b0acaa8 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:29.459385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3c7d-7cfc-4654-a7d4-652e9b0acaa8 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:29.461031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 458/925] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:29.470171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8b77-5c4f-4010-b73d-54e8f97d21f5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:29.470171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8b77-5c4f-4010-b73d-54e8f97d21f5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:29.470171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8b77-5c4f-4010-b73d-54e8f97d21f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:29.486516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-284c8b77-5c4f-4010-b73d-54e8f97d21f5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:29.487990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-284c8b77-5c4f-4010-b73d-54e8f97d21f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:29.528577 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:29.557644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef30293-ce68-46e4-a38d-a7e81d2013df None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:29.569400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 459/926] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:29.597969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2dfe-405b-4150-95f6-12c017eacad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:29.598609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2dfe-405b-4150-95f6-12c017eacad4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:29.598884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2dfe-405b-4150-95f6-12c017eacad4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:29.599153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2dfe-405b-4150-95f6-12c017eacad4 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:29.599759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae2dfe-405b-4150-95f6-12c017eacad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:29.601407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae2dfe-405b-4150-95f6-12c017eacad4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:29.608691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 468/927] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:34:29 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:34:29.838774 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c8b77-5c4f-4010-b73d-54e8f97d21f5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:29.941009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c8b77-5c4f-4010-b73d-54e8f97d21f5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:29.976438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:29.977423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:29.977956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:29.978457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:29.979266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:29.980831 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:29.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:29.991146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:34:29.992057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:29.995391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 460/928] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 8 13:34:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:30.003230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2dfe-405b-4150-95f6-12c017eacad4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:30.048993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:30.110343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2dfe-405b-4150-95f6-12c017eacad4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:30.145878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3af14-daac-4aae-aa37-6f4460571a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:30.146905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3af14-daac-4aae-aa37-6f4460571a8d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:30.147404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3af14-daac-4aae-aa37-6f4460571a8d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:30.147870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3af14-daac-4aae-aa37-6f4460571a8d None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:30.148691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3af14-daac-4aae-aa37-6f4460571a8d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:30.149186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3af14-daac-4aae-aa37-6f4460571a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:30.151019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3af14-daac-4aae-aa37-6f4460571a8d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:30.156734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 469/929] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:30.163364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:30.203954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:30.272347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220866c-0e68-4de1-a5c9-eeab5a9755f4 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:30.300281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb362ae-7531-40f8-962d-db98e1c7cdcf None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:30.300567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb362ae-7531-40f8-962d-db98e1c7cdcf None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:30.301008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb362ae-7531-40f8-962d-db98e1c7cdcf None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:30.306349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 470/930] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:30.312750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2df26-cc81-476d-872d-7f80fb971128 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:30.312750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2df26-cc81-476d-872d-7f80fb971128 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:30.312750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2df26-cc81-476d-872d-7f80fb971128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:30.313033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:30.313033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:30.313033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:30.313033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:30.313365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:30.313365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:30.313365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:30.318630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 461/931] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:30.324829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:30.329576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-39d2df26-cc81-476d-872d-7f80fb971128 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:30.331029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-39d2df26-cc81-476d-872d-7f80fb971128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:30.434163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 471/932] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 08 13:34:30.442931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:30.447033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f84ab5-9b8f-4c78-a329-12578168fa00 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:30.481968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cb1a7-67f8-44a0-bbe7-8a2c8b41e1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:30.482572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cb1a7-67f8-44a0-bbe7-8a2c8b41e1a6 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:30.482857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cb1a7-67f8-44a0-bbe7-8a2c8b41e1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:30.483127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cb1a7-67f8-44a0-bbe7-8a2c8b41e1a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:30.483795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cb1a7-67f8-44a0-bbe7-8a2c8b41e1a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:30.485148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cb1a7-67f8-44a0-bbe7-8a2c8b41e1a6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:30.587751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 462/933] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:34:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:34:30.753922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2df26-cc81-476d-872d-7f80fb971128 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:30.793218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:30.864411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2df26-cc81-476d-872d-7f80fb971128 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:30.907251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077afaa-47bd-4d3b-980b-f8ba116f4d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:30.908250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077afaa-47bd-4d3b-980b-f8ba116f4d2d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:30.908740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077afaa-47bd-4d3b-980b-f8ba116f4d2d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:30.909234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077afaa-47bd-4d3b-980b-f8ba116f4d2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:30.920468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077afaa-47bd-4d3b-980b-f8ba116f4d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-e077afaa-47bd-4d3b-980b-f8ba116f4d2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:30.925631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:30.926649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:30.927737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:30.933932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 472/934] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 13:34:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:30.936519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cb1a7-67f8-44a0-bbe7-8a2c8b41e1a6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:31.053294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cb1a7-67f8-44a0-bbe7-8a2c8b41e1a6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:31.089166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:31.089787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.090069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.090371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.091294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:31.091597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:31.092960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:31.095283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d83820-869d-4db0-b0d0-48cfd395b682 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.095706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d83820-869d-4db0-b0d0-48cfd395b682 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.096607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d83820-869d-4db0-b0d0-48cfd395b682 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.098267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 473/935] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:31.102398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 463/936] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 13:34:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:31.103382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5742c4-35e2-47f8-8f49-895e8f8ec25f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.103607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5742c4-35e2-47f8-8f49-895e8f8ec25f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.103874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5742c4-35e2-47f8-8f49-895e8f8ec25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.108612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:31.125877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ad5742c4-35e2-47f8-8f49-895e8f8ec25f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:31.127442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ad5742c4-35e2-47f8-8f49-895e8f8ec25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:31.132468 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:31.208220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 474/937] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:31.222590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95cee9-df2b-49a7-a269-790a443375ca None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:31.258712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:31.259460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.259742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.260015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.271370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:31.272870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:31.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:31.291066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:31.291980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:31.291980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:31.291980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.294818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 464/938] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:31.296947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5742c4-35e2-47f8-8f49-895e8f8ec25f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:31.322463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:31.441527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5742c4-35e2-47f8-8f49-895e8f8ec25f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:31.479311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:31.479943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.480222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.480544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.481206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:31.481572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:31.483001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:31.488990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:31.502016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 475/939] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:31.503297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:31.583580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:31.606566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:31.618627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246541d-614b-4bd5-a579-f5108adc0645 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:31.624955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d988c8-8a97-4b5c-bf63-d14ec4d97e6c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:31.664067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:31.665892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.666452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.666582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:31.667135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.667243 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.667451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.667731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.668437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:31.669851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:31.673280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING keystone.server.flask.common [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 13:34:31.676961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:31.678773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:31.692485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:31.693447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:31.694681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:31.697343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 476/940] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:34:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:31.703452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:31.747719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.sql.core [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:34:31.747719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 13:34:31.747719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [parameters: {'id': '87502ea96d40450284c9e9bee7ea4d60', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 13:34:31.747719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88255) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 87502ea96d40450284c9e9bee7ea4d60.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 87502ea96d40450284c9e9bee7ea4d60. Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:34:31.770737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application result.read() Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 08 13:34:31.771640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application yield resource Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise self.value Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application session.commit() Sep 08 13:34:31.772624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:34:31.773538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:34:31.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application result.read() Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:34:31.781565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [parameters: {'id': '87502ea96d40450284c9e9bee7ea4d60', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:31.784878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:31.787204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 87502ea96d40450284c9e9bee7ea4d60. Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 465/941] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 8 13:34:31 2022] POST /identity/v3/projects => generated 262 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:31.791061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:31.837998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7953fe-4c81-4081-9f47-ba98a5de3c86 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:31.875390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c115dba-fab1-4ca0-b530-2e8cccb2005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:31.875959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c115dba-fab1-4ca0-b530-2e8cccb2005c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.876237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c115dba-fab1-4ca0-b530-2e8cccb2005c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.876516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c115dba-fab1-4ca0-b530-2e8cccb2005c None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.877397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c115dba-fab1-4ca0-b530-2e8cccb2005c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:31.877578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c115dba-fab1-4ca0-b530-2e8cccb2005c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:31.879022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c115dba-fab1-4ca0-b530-2e8cccb2005c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:31.895463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:31.903811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 477/942] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 13:34:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:31.903811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6999f4-569c-4296-be83-aaaafa87bcd0 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.903811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6999f4-569c-4296-be83-aaaafa87bcd0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.903811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6999f4-569c-4296-be83-aaaafa87bcd0 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.906277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 478/943] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:31.907350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6999f4-569c-4296-be83-aaaafa87bcd0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:31.926404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5d39d-3d20-4669-af92-b0bb1e672f5e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:31.955711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:31.964485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ce500-44d9-4b03-b9a2-aefee87061e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:31.965118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ce500-44d9-4b03-b9a2-aefee87061e8 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.965397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ce500-44d9-4b03-b9a2-aefee87061e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.966217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ce500-44d9-4b03-b9a2-aefee87061e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:31.972727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ce500-44d9-4b03-b9a2-aefee87061e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:31.974324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ce500-44d9-4b03-b9a2-aefee87061e8 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f98ce500-44d9-4b03-b9a2-aefee87061e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:31.987816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:31.988798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:31.989713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:31.989713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:31.989713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:31.989713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:31.989713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:31.989713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:31.989713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:31.989713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:31.991632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 466/944] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:31.992922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86ad1-1aca-4f4f-99b8-9607a4a67002 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:31.993732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86ad1-1aca-4f4f-99b8-9607a4a67002 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:31.994003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86ad1-1aca-4f4f-99b8-9607a4a67002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.015901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-10c86ad1-1aca-4f4f-99b8-9607a4a67002 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:32.017455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-10c86ad1-1aca-4f4f-99b8-9607a4a67002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:32.022802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6999f4-569c-4296-be83-aaaafa87bcd0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.058498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.059125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.059400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.059666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.070311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.072467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:32.087694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:32.088715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:32.095294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:32.095294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:32.095294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:32.095471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 467/945] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:32.095471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c86ad1-1aca-4f4f-99b8-9607a4a67002 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.098287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 479/946] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:32.099490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.161869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:32.218324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb33d7d-c724-4a0d-b07e-a1fa910bb3d1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.220136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c86ad1-1aca-4f4f-99b8-9607a4a67002 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.238802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:32.259548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.260197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.260498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.261036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.261493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:32.261800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.263261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.267265 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.268121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.268607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.269101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.269949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:32.270490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.272068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.288496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 468/947] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:32.289027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 480/948] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:32.307808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.371961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.433726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64584ff-0b84-44a7-ba16-c2c8a37d46de None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.476216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.476874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.477159 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.477437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.486614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.488695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.491820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3f1b2-b6ff-4153-a87f-069c929e0c86 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:32.502187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:32.503139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:32.504059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:32.504059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:32.504059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:32.504059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:32.504059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:32.504059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:32.504059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:32.504059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:32.507871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:32.508214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 469/949] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:32.509502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.542170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.542827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.543111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.543386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.543980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.545356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:32.554679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:32.556675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:32.557799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 481/950] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:32.563954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:32.565880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.653047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8093-4332-467b-94db-7a026cdc1ead None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.691847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.692563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.692917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.693262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.694323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:32.695182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.697149 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.704553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815c61c-00ec-468c-99d6-195f7be12e23 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.713799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 470/951] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:32.723117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.745357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.745975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.746285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.747670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.747670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:32.747670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.748909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.754610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 482/952] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:32.760444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.813313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:32.853055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:32.860828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8974a-8807-4193-b528-bd5c8a5b5cf6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.883287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49b45c-4ef0-4f48-9a00-ece5bd428d30 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:32.897308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc5c6a-65ec-48a5-b6f6-5feb9cca4e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.897892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc5c6a-65ec-48a5-b6f6-5feb9cca4e87 None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.898272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc5c6a-65ec-48a5-b6f6-5feb9cca4e87 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.898550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc5c6a-65ec-48a5-b6f6-5feb9cca4e87 None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/df727be6307948c49affa47e601f989c/roles/1ab14f52bb114cefadb70cd2a83acf19` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.901758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc5c6a-65ec-48a5-b6f6-5feb9cca4e87 None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=df727be6307948c49affa47e601f989c, role_id=1ab14f52bb114cefadb70cd2a83acf19)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.903091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc5c6a-65ec-48a5-b6f6-5feb9cca4e87 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.924196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:32.924816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.925099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.925375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.926011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:32.926345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:32.927745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:32.933114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 483/953] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:32.938448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:32.966306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 471/954] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:34:32 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/df727be6307948c49affa47e601f989c/roles/1ab14f52bb114cefadb70cd2a83acf19 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:34:32.967584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137609-b1af-4e4e-87a6-394f564b9b99 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.967816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137609-b1af-4e4e-87a6-394f564b9b99 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.968095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137609-b1af-4e4e-87a6-394f564b9b99 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.974045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 472/955] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:32.975968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a4d9f-4a41-4a04-991e-9ffe13441565 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:32.976261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a4d9f-4a41-4a04-991e-9ffe13441565 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:32.976541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a4d9f-4a41-4a04-991e-9ffe13441565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:32.999322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0f5a4d9f-4a41-4a04-991e-9ffe13441565 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:33.001049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0f5a4d9f-4a41-4a04-991e-9ffe13441565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:33.084844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dcfde-2609-4221-aa76-010b90939fac None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:33.093254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:33.095707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 473/956] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:33.132465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34ca0e-a4fc-4cf5-b3b1-081103316c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:33.133461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34ca0e-a4fc-4cf5-b3b1-081103316c12 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:33.134532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34ca0e-a4fc-4cf5-b3b1-081103316c12 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:33.135024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34ca0e-a4fc-4cf5-b3b1-081103316c12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:33.136001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34ca0e-a4fc-4cf5-b3b1-081103316c12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:33.137707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34ca0e-a4fc-4cf5-b3b1-081103316c12 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:33.238506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 484/957] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:34:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:34:33.378971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a4d9f-4a41-4a04-991e-9ffe13441565 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:33.482340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:33.497866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a4d9f-4a41-4a04-991e-9ffe13441565 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:33.541158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:33.541797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:33.542074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:33.542383 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:33.542963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:33.544316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:33.553238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:33.554484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:33.555686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 474/958] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:33.563541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34ca0e-a4fc-4cf5-b3b1-081103316c12 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:33.687318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34ca0e-a4fc-4cf5-b3b1-081103316c12 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:33.723861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66af20-06eb-4f2f-b04f-f05f7ea599f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:33.724853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66af20-06eb-4f2f-b04f-f05f7ea599f5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:33.725335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66af20-06eb-4f2f-b04f-f05f7ea599f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:33.725808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66af20-06eb-4f2f-b04f-f05f7ea599f5 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:33.726698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66af20-06eb-4f2f-b04f-f05f7ea599f5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:33.727204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66af20-06eb-4f2f-b04f-f05f7ea599f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:33.729356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66af20-06eb-4f2f-b04f-f05f7ea599f5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:33.735138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 485/959] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:33.744768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:33.854406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aed0a8-81a7-4af8-a28a-da3cacaa8bd6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:33.890090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239e1ff-04f6-460b-9af8-c427f6e8edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:33.890737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239e1ff-04f6-460b-9af8-c427f6e8edd5 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:33.891022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239e1ff-04f6-460b-9af8-c427f6e8edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:33.891298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239e1ff-04f6-460b-9af8-c427f6e8edd5 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:33.891931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239e1ff-04f6-460b-9af8-c427f6e8edd5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:33.892234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239e1ff-04f6-460b-9af8-c427f6e8edd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:33.893593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239e1ff-04f6-460b-9af8-c427f6e8edd5 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:33.898555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 475/960] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:34.703217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab86c2-399f-491b-bb5d-51c564e0efcb None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:34.703729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab86c2-399f-491b-bb5d-51c564e0efcb None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:34.703807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab86c2-399f-491b-bb5d-51c564e0efcb None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:34.705427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 486/961] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:34.710773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c198ab-c138-488d-be8a-c7bb2d68b9ca None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:34.711197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c198ab-c138-488d-be8a-c7bb2d68b9ca None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:34.711401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c198ab-c138-488d-be8a-c7bb2d68b9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:34.731557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-65c198ab-c138-488d-be8a-c7bb2d68b9ca None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:34.732766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-65c198ab-c138-488d-be8a-c7bb2d68b9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:34.735741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:34.806563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 476/962] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:35.100314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab86c2-399f-491b-bb5d-51c564e0efcb None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:35.128345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:35.216408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab86c2-399f-491b-bb5d-51c564e0efcb None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:35.255826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:35.256449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:35.256721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:35.256999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:35.257660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:35.259177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:35.261778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING keystone.server.flask.common [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 13:34:35.322828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.sql.core [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:34:35.322828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 13:34:35.322828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [parameters: {'user_id': 'a970229edd3e4b30aa9da12cc8796529', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 13:34:35.322828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88256) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:34:35.327123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c8440-1ae3-4eb8-9932-5ed53395711c None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:35.327412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c8440-1ae3-4eb8-9932-5ed53395711c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:35.327686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c8440-1ae3-4eb8-9932-5ed53395711c None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:35.329301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 477/963] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result.read() Sep 08 13:34:35.330130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:34:35.331168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application yield resource Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise self.value Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application session.commit() Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:34:35.332219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:34:35.333608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:34:35.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application result.read() Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:34:35.338069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a970229edd3e4b30aa9da12cc8796529', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:35.341748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 08 13:34:35.343937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 08 13:34:35.347177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 08 13:34:35.347177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 08 13:34:35.347177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:34:35.347177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:34:35.347177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 13:34:35.347177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:35.347177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 487/964] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 13:34:35 2022] POST /identity/v3/users => generated 160 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:35.348079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f253a20-d7a5-483e-b140-f0bdfeb9b0c6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:35.348079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f253a20-d7a5-483e-b140-f0bdfeb9b0c6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:35.348079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f253a20-d7a5-483e-b140-f0bdfeb9b0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:35.348531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:35.368285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-4f253a20-d7a5-483e-b140-f0bdfeb9b0c6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:35.369424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4f253a20-d7a5-483e-b140-f0bdfeb9b0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:35.452062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 478/965] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:35.472215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:35.491852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ef526-0e7d-433d-bd71-886ef365fc49 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:35.531453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8b766-682b-4f71-9813-7e35aeb86dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:35.532478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8b766-682b-4f71-9813-7e35aeb86dd6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:35.532959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8b766-682b-4f71-9813-7e35aeb86dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:35.533428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8b766-682b-4f71-9813-7e35aeb86dd6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:35.542159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8b766-682b-4f71-9813-7e35aeb86dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:35.544053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8b766-682b-4f71-9813-7e35aeb86dd6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-2cc8b766-682b-4f71-9813-7e35aeb86dd6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:35.556070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:35.557051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:34:35.557998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:35.560670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 488/966] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:35.569499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f253a20-d7a5-483e-b140-f0bdfeb9b0c6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:35.605516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977e01-dcef-4452-b904-e657aeb4afdf None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:35.606166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977e01-dcef-4452-b904-e657aeb4afdf None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:35.606645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977e01-dcef-4452-b904-e657aeb4afdf None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:35.611308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 489/967] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:35.616605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242589e9-8833-4340-8332-9a7702980580 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:35.617158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242589e9-8833-4340-8332-9a7702980580 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:35.617631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242589e9-8833-4340-8332-9a7702980580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:35.641063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-242589e9-8833-4340-8332-9a7702980580 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:35.642838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-242589e9-8833-4340-8332-9a7702980580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:35.654278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:35.690508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f253a20-d7a5-483e-b140-f0bdfeb9b0c6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:35.725684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 490/968] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:35.726784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242589e9-8833-4340-8332-9a7702980580 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:35.731663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:35.732280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:35.732562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:35.732837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:35.733543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:35.733849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:35.735319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:35.782230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 479/969] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 13:34:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:35.834964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:35.861522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242589e9-8833-4340-8332-9a7702980580 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:35.902488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4a05-dfdf-4578-944a-0c7f83423afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:35.903535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4a05-dfdf-4578-944a-0c7f83423afd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:35.904037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4a05-dfdf-4578-944a-0c7f83423afd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:35.904508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4a05-dfdf-4578-944a-0c7f83423afd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:35.917743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a4a05-dfdf-4578-944a-0c7f83423afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-512a4a05-dfdf-4578-944a-0c7f83423afd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:35.923429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:35.924544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:35.925405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:35.928215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 491/970] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 13:34:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:35.934709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.005039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4a05-dfdf-4578-944a-0c7f83423afd None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.057382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62734a37-5c83-409f-aba3-dafd4a8a0ad3 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.089360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:36.097971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.104761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.104761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.104761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.104761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:36.104761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.104761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.111816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 480/971] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 13:34:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:36.117771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.152567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4a05-dfdf-4578-944a-0c7f83423afd None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.186082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f1cb9-b068-4ba0-9232-bb9c314ec711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.186738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f1cb9-b068-4ba0-9232-bb9c314ec711 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.187018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f1cb9-b068-4ba0-9232-bb9c314ec711 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.187292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f1cb9-b068-4ba0-9232-bb9c314ec711 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.187877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f1cb9-b068-4ba0-9232-bb9c314ec711 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.189244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f1cb9-b068-4ba0-9232-bb9c314ec711 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.191766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-861f1cb9-b068-4ba0-9232-bb9c314ec711 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:36.198887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:36.199920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:36.200793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:36.202286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 492/972] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:36.204008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd74779-52fc-4267-864b-d25c73d57d78 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.204008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd74779-52fc-4267-864b-d25c73d57d78 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.204439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd74779-52fc-4267-864b-d25c73d57d78 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.205851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 493/973] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:36.206574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd74779-52fc-4267-864b-d25c73d57d78 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.223504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152234d-0e4e-406e-aeb2-432bf4642ab9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.257500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0640b-a755-4377-8b04-55afdb0f365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.258203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0640b-a755-4377-8b04-55afdb0f365d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.259031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0640b-a755-4377-8b04-55afdb0f365d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.259309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0640b-a755-4377-8b04-55afdb0f365d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.269856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0640b-a755-4377-8b04-55afdb0f365d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.271366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0640b-a755-4377-8b04-55afdb0f365d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-90d0640b-a755-4377-8b04-55afdb0f365d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:36.286256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:36.287364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:36.288751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:36.288751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:36.288751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:36.288751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 481/974] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:36.290630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1639a0-bd55-4190-93f8-9ee2519d42ab None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.291002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1639a0-bd55-4190-93f8-9ee2519d42ab None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.291285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1639a0-bd55-4190-93f8-9ee2519d42ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.308933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-cb1639a0-bd55-4190-93f8-9ee2519d42ab None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:36.309239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd74779-52fc-4267-864b-d25c73d57d78 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.310165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cb1639a0-bd55-4190-93f8-9ee2519d42ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:36.344641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.345250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.345529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.345801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.346535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:36.346643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:36.347128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.348198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.352923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 494/975] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:36.353703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.384195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 482/976] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:36.385026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1639a0-bd55-4190-93f8-9ee2519d42ab None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.427002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:36.468272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d795e2-5641-4bc3-a3d0-b55746b5029b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.495654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1639a0-bd55-4190-93f8-9ee2519d42ab None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.507864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.509191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.509478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.509752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.510444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:36.510754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.512131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.524210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 495/977] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:36.529750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.537111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.537732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.538451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.538736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.539373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:36.539706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.541772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.547165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 483/978] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:36.552738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.591044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:36.643718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649ded0-eb2d-48d8-bcec-dd2feb4f77ae None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.660716 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ce019-6f70-4653-89c8-b5c2a13b72a6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.667886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:36.679006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04746829-5526-40fa-a631-cfea782b076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.679156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04746829-5526-40fa-a631-cfea782b076b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.679479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04746829-5526-40fa-a631-cfea782b076b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.679748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04746829-5526-40fa-a631-cfea782b076b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.687283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04746829-5526-40fa-a631-cfea782b076b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.688705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04746829-5526-40fa-a631-cfea782b076b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.694504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.695112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.695391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.695660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.696415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] RBAC: Adding query filter params (interface=admin, service_id=2bbc0564ebb94884a14a9f420a5553db, region_id=RegionOne) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:36.696721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.698070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-04746829-5526-40fa-a631-cfea782b076b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:36.701153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:36.702096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:36.708173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 496/979] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:36.708945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 484/980] 10.222.0.25 () {56 vars in 1249 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/endpoints?service_id=2bbc0564ebb94884a14a9f420a5553db&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:36.709042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04746829-5526-40fa-a631-cfea782b076b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.709126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.804949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:36.806970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04746829-5526-40fa-a631-cfea782b076b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.808069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7bfd-4acc-4558-ab08-5008334cc73b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:36.842820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b0799-fb4b-454b-94d3-a4e6974efbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.843023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:36.843442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b0799-fb4b-454b-94d3-a4e6974efbc9 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.843547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:36.843718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b0799-fb4b-454b-94d3-a4e6974efbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.843837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:36.843988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b0799-fb4b-454b-94d3-a4e6974efbc9 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.844079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:36.844629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b0799-fb4b-454b-94d3-a4e6974efbc9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.844734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:36.845011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:36.845988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b0799-fb4b-454b-94d3-a4e6974efbc9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.846441 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:36.863932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 485/981] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:36.866900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.908569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 497/982] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:34:36 2022] POST /identity/v3/services => generated 242 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:36.909402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b0799-fb4b-454b-94d3-a4e6974efbc9 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:36.962230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:36.974084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d209f-431d-4e17-8ec8-6f797be596c9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:37.020755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83ce4-70df-45f6-b9ca-f787e6973a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:37.021382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83ce4-70df-45f6-b9ca-f787e6973a3f None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:37.021666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83ce4-70df-45f6-b9ca-f787e6973a3f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:37.021936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83ce4-70df-45f6-b9ca-f787e6973a3f None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:37.022592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83ce4-70df-45f6-b9ca-f787e6973a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:37.024073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83ce4-70df-45f6-b9ca-f787e6973a3f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:37.027710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b0799-fb4b-454b-94d3-a4e6974efbc9 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:37.041038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 486/983] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:34:37.071979 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f81ca-e1b3-4a6e-84a9-6790da9b924c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:37.072986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f81ca-e1b3-4a6e-84a9-6790da9b924c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:37.073446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f81ca-e1b3-4a6e-84a9-6790da9b924c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:37.073913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f81ca-e1b3-4a6e-84a9-6790da9b924c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:37.162696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f81ca-e1b3-4a6e-84a9-6790da9b924c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:37.164165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f81ca-e1b3-4a6e-84a9-6790da9b924c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:37.216139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:37.275414 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 498/984] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:36 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:37.282229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83ce4-70df-45f6-b9ca-f787e6973a3f None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:37.302405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:37.384466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83ce4-70df-45f6-b9ca-f787e6973a3f None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:37.419423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:37.420232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:37.420516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:37.420788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:37.431161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:37.432612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:37.447323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:37.448347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:37.449278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:37.449278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:37.449278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:37.453067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 487/985] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:37.454465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f81ca-e1b3-4a6e-84a9-6790da9b924c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:37.572426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f81ca-e1b3-4a6e-84a9-6790da9b924c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:37.612047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:37.612657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:37.612939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:37.613215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:37.613865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:37.614198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:37.615660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:37.627503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 499/986] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:37.633800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:37.710906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:37.744757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aafd00-19c8-4834-82a2-b7280ba020f5 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:37.780568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:37.781207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:37.781491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:37.781783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:37.788957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:37.790536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:37.802864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:37.803801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:37.804726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 488/987] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:37.809884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:37.817667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:37.918884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50e9ae-ed4d-4dea-8166-174579720d4d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:37.955359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df512b83-5419-482d-9561-e145c6702676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:37.955976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df512b83-5419-482d-9561-e145c6702676 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:37.956255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df512b83-5419-482d-9561-e145c6702676 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:37.956521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df512b83-5419-482d-9561-e145c6702676 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:37.957207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df512b83-5419-482d-9561-e145c6702676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:37.957509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df512b83-5419-482d-9561-e145c6702676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:37.958982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df512b83-5419-482d-9561-e145c6702676 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:37.971740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 500/988] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:37.977692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:38.078952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b01bd1-9424-4f1b-ac29-a058fa4dac2b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:38.106496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:38.112656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8af29-c8dd-44fe-9f43-c6168e99addf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:38.113262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8af29-c8dd-44fe-9f43-c6168e99addf None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:38.113545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8af29-c8dd-44fe-9f43-c6168e99addf None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:38.113822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8af29-c8dd-44fe-9f43-c6168e99addf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:38.114690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8af29-c8dd-44fe-9f43-c6168e99addf None admin] RBAC: Adding query filter params (role.id=1ab14f52bb114cefadb70cd2a83acf19, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=df727be6307948c49affa47e601f989c) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:38.116873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8af29-c8dd-44fe-9f43-c6168e99addf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:38.116873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8af29-c8dd-44fe-9f43-c6168e99addf None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:38.130179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 489/989] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:37 2022] GET /identity/v3/role_assignments?user.id=df727be6307948c49affa47e601f989c&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=1ab14f52bb114cefadb70cd2a83acf19 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:39.179468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f81dce-2a45-45ca-beb8-b68c23e843c0 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:39.180017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f81dce-2a45-45ca-beb8-b68c23e843c0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:39.180098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f81dce-2a45-45ca-beb8-b68c23e843c0 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:39.181558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 501/990] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:39.186505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a30b8-3846-4018-88e6-d5dc6528df66 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:39.186890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a30b8-3846-4018-88e6-d5dc6528df66 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:39.187058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a30b8-3846-4018-88e6-d5dc6528df66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:39.205687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-331a30b8-3846-4018-88e6-d5dc6528df66 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:39.206910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-331a30b8-3846-4018-88e6-d5dc6528df66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:39.251469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3fb6-da2d-4fb8-8e59-9cc562bb72a6 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:39.256290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3fb6-da2d-4fb8-8e59-9cc562bb72a6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:39.256290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3fb6-da2d-4fb8-8e59-9cc562bb72a6 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:39.256290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 502/991] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:39.258807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa72e6-1ef7-47bb-bc16-f56ff79f107a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:39.259092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa72e6-1ef7-47bb-bc16-f56ff79f107a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:39.259367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa72e6-1ef7-47bb-bc16-f56ff79f107a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:39.277229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 490/992] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:39.277446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-25fa72e6-1ef7-47bb-bc16-f56ff79f107a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:39.278686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-25fa72e6-1ef7-47bb-bc16-f56ff79f107a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:39.343882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:39.353172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 503/993] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:39.562598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a30b8-3846-4018-88e6-d5dc6528df66 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:39.622016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa72e6-1ef7-47bb-bc16-f56ff79f107a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:39.673065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a30b8-3846-4018-88e6-d5dc6528df66 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:39.698816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:39.709967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:39.710623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:39.710908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:39.711232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:39.711821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:39.713178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:39.722734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:39.723729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:39.724586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 491/994] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:39.730277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:39.733948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa72e6-1ef7-47bb-bc16-f56ff79f107a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:39.770658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:39.771415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:39.773012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:39.773012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:39.773012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:39.773943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:39.782961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:39.783986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:39.784838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 504/995] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 13:34:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:39.790460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:39.843522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da500d0-ad5f-463f-a219-659920ccd7ea None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:39.852367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:39.879614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:39.880224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:39.880499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:39.880777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:39.881573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] RBAC: Adding query filter params (name=compute) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:39.881873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:39.883878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:39.889126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 492/996] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:39.894639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:39.899937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fb737-f5ad-41ab-b57e-e7c8d91954da None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:39.933651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:39.934566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:39.934999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:39.935281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:39.935913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:39.936213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:39.937573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:39.947543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 505/997] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:39.949279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:39.949436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:40.005723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dda2c-1b2b-4364-a60e-63d527de0cdb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:40.043167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:40.044225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:40.044508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:40.044938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:40.045573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:40.046041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:40.047773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:40.057883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 493/998] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:40.058049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a4ed-6065-4a65-aa5a-bafe5a09716c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:40.058958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:40.081298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:40.098864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:40.098864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:40.098864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:40.098864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:40.099226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:40.099226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:40.099226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:40.102671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 506/999] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 13:34:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:40.108174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:40.173106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07faab-9785-430d-9f0e-52aa28173e52 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:40.190165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:40.210702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:40.211875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:40.212287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:40.212557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:40.213291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] RBAC: Adding query filter params (interface=public, service_id=20b182797dd2486580fe41e3399e3628, region_id=RegionOne) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:40.213603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:40.215044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:40.216638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acd1b2-0e54-4f44-8a69-ab68442fa31b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:40.228068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 494/1000] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 13:34:40 2022] GET /identity/v3/endpoints?service_id=20b182797dd2486580fe41e3399e3628&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:40.229073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:40.257928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513fb6b-dd5d-4ff7-a460-ee993ae6f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:40.258577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513fb6b-dd5d-4ff7-a460-ee993ae6f52c None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:40.258858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513fb6b-dd5d-4ff7-a460-ee993ae6f52c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:40.259128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513fb6b-dd5d-4ff7-a460-ee993ae6f52c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:40.259787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513fb6b-dd5d-4ff7-a460-ee993ae6f52c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:40.261208 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513fb6b-dd5d-4ff7-a460-ee993ae6f52c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:40.341043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 507/1001] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:34:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:34:40.342647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58a9ab-157a-48b7-b186-6d42d5399e05 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:40.342934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58a9ab-157a-48b7-b186-6d42d5399e05 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:40.343209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58a9ab-157a-48b7-b186-6d42d5399e05 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:40.344513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 508/1002] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:40.350916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d397032-df82-4591-9d97-c4d7fead1ff8 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:40.351230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d397032-df82-4591-9d97-c4d7fead1ff8 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:40.351410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d397032-df82-4591-9d97-c4d7fead1ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:40.370521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53d6d-18c6-4be2-b860-64f48b83c1fb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:40.373180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-4d397032-df82-4591-9d97-c4d7fead1ff8 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:40.379665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4d397032-df82-4591-9d97-c4d7fead1ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:40.417189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f786d-dff0-4890-9656-c7363adbf074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:40.417818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f786d-dff0-4890-9656-c7363adbf074 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:40.418096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f786d-dff0-4890-9656-c7363adbf074 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:40.419476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f786d-dff0-4890-9656-c7363adbf074 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:40.420152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f786d-dff0-4890-9656-c7363adbf074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:40.421546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f786d-dff0-4890-9656-c7363adbf074 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:40.427200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:40.428593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 495/1003] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:34:40 2022] GET /identity/v3/services => generated 1727 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:34:40.458896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 509/1004] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:40.728160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f786d-dff0-4890-9656-c7363adbf074 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:40.771936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:40.829918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f786d-dff0-4890-9656-c7363adbf074 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:40.867006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:40.868201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:40.868477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:40.868748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] PATH_INFO: `/v3/services/image` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:40.869330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:40.870715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:40.879947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:40.882352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:40.883902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 496/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 8 13:34:40 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:40.887897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d397032-df82-4591-9d97-c4d7fead1ff8 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:40.991063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d397032-df82-4591-9d97-c4d7fead1ff8 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:41.029024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c569a-ed6f-4636-8ba9-cf92caa25677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:41.029637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c569a-ed6f-4636-8ba9-cf92caa25677 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:41.029918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c569a-ed6f-4636-8ba9-cf92caa25677 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:41.030228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c569a-ed6f-4636-8ba9-cf92caa25677 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:41.030864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c569a-ed6f-4636-8ba9-cf92caa25677 None admin] RBAC: Adding query filter params (name=image) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:41.031176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c569a-ed6f-4636-8ba9-cf92caa25677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:41.032539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c569a-ed6f-4636-8ba9-cf92caa25677 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:41.037432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 510/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:41.044179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:41.145942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197df280-38f9-4272-af3c-72142ab05696 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:41.149024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:41.182121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c0e64-9060-462c-8f28-8ad362f6db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:41.182846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c0e64-9060-462c-8f28-8ad362f6db41 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:41.183194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c0e64-9060-462c-8f28-8ad362f6db41 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:41.183535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c0e64-9060-462c-8f28-8ad362f6db41 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:41.184237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c0e64-9060-462c-8f28-8ad362f6db41 None admin] RBAC: Adding query filter params (type=image) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:41.184608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c0e64-9060-462c-8f28-8ad362f6db41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:41.186030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c0e64-9060-462c-8f28-8ad362f6db41 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:41.198254 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 497/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:42.568704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7fe1a-66b4-4347-a009-f14739a809f0 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:42.569237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7fe1a-66b4-4347-a009-f14739a809f0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:42.569237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7fe1a-66b4-4347-a009-f14739a809f0 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:42.571395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 511/1008] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:42.576687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e58b1f-2dc7-4a4e-9504-8fd3aa3365ad None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:42.577079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e58b1f-2dc7-4a4e-9504-8fd3aa3365ad None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:42.577270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e58b1f-2dc7-4a4e-9504-8fd3aa3365ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:42.598583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-32e58b1f-2dc7-4a4e-9504-8fd3aa3365ad None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:42.599784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-32e58b1f-2dc7-4a4e-9504-8fd3aa3365ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:42.628442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b3d3-2619-42f6-9332-1e04b9b5a09f None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:42.628738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b3d3-2619-42f6-9332-1e04b9b5a09f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:42.628989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b3d3-2619-42f6-9332-1e04b9b5a09f None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:42.630425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 512/1009] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:42.635986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9fe58-67e0-45d5-be24-c0a196421939 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:42.636331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9fe58-67e0-45d5-be24-c0a196421939 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:42.636620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9fe58-67e0-45d5-be24-c0a196421939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:42.654795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-3cc9fe58-67e0-45d5-be24-c0a196421939 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:42.656007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3cc9fe58-67e0-45d5-be24-c0a196421939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:42.675605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 498/1010] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:42 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:42.694990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:42.730274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 513/1011] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:42 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:42.960150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e58b1f-2dc7-4a4e-9504-8fd3aa3365ad None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:42.996061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9fe58-67e0-45d5-be24-c0a196421939 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:43.061906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:43.065947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e58b1f-2dc7-4a4e-9504-8fd3aa3365ad None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:43.099793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9fe58-67e0-45d5-be24-c0a196421939 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:43.106588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43c828-ed62-4adc-9010-aedc9b483983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:43.107226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c828-ed62-4adc-9010-aedc9b483983 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.107510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c828-ed62-4adc-9010-aedc9b483983 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.107785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c828-ed62-4adc-9010-aedc9b483983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.109296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43c828-ed62-4adc-9010-aedc9b483983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:43.111153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43c828-ed62-4adc-9010-aedc9b483983 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:43.113228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 499/1012] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:43.119425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43c828-ed62-4adc-9010-aedc9b483983 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:43.137320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:43.138328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.138856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.139324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.140080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:43.141793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:43.151413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:43.152448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:43.153366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 514/1013] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:43.158292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:43.195006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:43.229157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43c828-ed62-4adc-9010-aedc9b483983 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:43.268804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2e406-17c6-476e-bcc8-c56d0dd8d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:43.269807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e406-17c6-476e-bcc8-c56d0dd8d5c9 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.270090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e406-17c6-476e-bcc8-c56d0dd8d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.270396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e406-17c6-476e-bcc8-c56d0dd8d5c9 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.271069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2e406-17c6-476e-bcc8-c56d0dd8d5c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:43.272534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2e406-17c6-476e-bcc8-c56d0dd8d5c9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:43.273752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3333d5-d200-45a2-8f45-1b4fa11e8d53 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:43.311556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:43.312168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.312447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.312719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.313382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:43.313684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:43.315115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:43.320670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 515/1014] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:43.326444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:43.347294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 500/1015] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:34:43 2022] POST /identity/v3/projects => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:34:43.373467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e6d7-692d-4026-a963-dd1303643db7 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.373817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e6d7-692d-4026-a963-dd1303643db7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.374342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e6d7-692d-4026-a963-dd1303643db7 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.375751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 501/1016] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:43.381540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053876b-eb7e-485c-aff6-d29a3e11e329 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.381867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053876b-eb7e-485c-aff6-d29a3e11e329 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.382104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053876b-eb7e-485c-aff6-d29a3e11e329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.403130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6053876b-eb7e-485c-aff6-d29a3e11e329 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:43.404442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6053876b-eb7e-485c-aff6-d29a3e11e329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:43.421798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:43.444395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:43.448601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63667ed-f0ae-430e-bf0d-5f3386d07938 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:43.482809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 502/1017] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:43 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:43.483955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:43.484578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.484854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.485123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.485789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:43.486093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:43.487542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:43.499203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 516/1018] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 13:34:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:43.500436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053876b-eb7e-485c-aff6-d29a3e11e329 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:43.621601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053876b-eb7e-485c-aff6-d29a3e11e329 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:43.658594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d43bd-26c9-4bc2-ba06-cc938aa7417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:43.659209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d43bd-26c9-4bc2-ba06-cc938aa7417f None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.659491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d43bd-26c9-4bc2-ba06-cc938aa7417f None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.659765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d43bd-26c9-4bc2-ba06-cc938aa7417f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.660447 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d43bd-26c9-4bc2-ba06-cc938aa7417f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:43.661856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d43bd-26c9-4bc2-ba06-cc938aa7417f None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:43.734382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 503/1019] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:34:43 2022] POST /identity/v3/endpoints => generated 334 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:34:43.758521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:43.870807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c0f17-24f9-4e5f-a4c9-f8b776a02fd8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:43.918502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fc85a-d610-4827-98a2-bc63679aabd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:43.919118 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fc85a-d610-4827-98a2-bc63679aabd9 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:43.919431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fc85a-d610-4827-98a2-bc63679aabd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:43.919667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fc85a-d610-4827-98a2-bc63679aabd9 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:43.920312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fc85a-d610-4827-98a2-bc63679aabd9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:43.921677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fc85a-d610-4827-98a2-bc63679aabd9 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:44.033808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 517/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:34:43 2022] POST /identity/v3/services => generated 242 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:34:45.597094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621de7e-70d2-4f42-9329-78e5b7d44bcf None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:45.597610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621de7e-70d2-4f42-9329-78e5b7d44bcf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:45.597690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621de7e-70d2-4f42-9329-78e5b7d44bcf None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:45.599187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 504/1021] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:45.604499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690ebba-f645-44d5-b243-250d24957370 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:45.604913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690ebba-f645-44d5-b243-250d24957370 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:45.605092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690ebba-f645-44d5-b243-250d24957370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:45.624317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-2690ebba-f645-44d5-b243-250d24957370 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:45.625523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2690ebba-f645-44d5-b243-250d24957370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:45.639814 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:45.704107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 518/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:45.969536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621de7e-70d2-4f42-9329-78e5b7d44bcf None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:45.973410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:46.055801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13b413-b29c-4a52-a133-a79769e8cac1 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.056115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13b413-b29c-4a52-a133-a79769e8cac1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.056657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13b413-b29c-4a52-a133-a79769e8cac1 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.057840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 519/1023] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:46.062533 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625d52-c57c-421a-93c9-2c1ddac4209b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.062816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625d52-c57c-421a-93c9-2c1ddac4209b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.063094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625d52-c57c-421a-93c9-2c1ddac4209b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.077854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621de7e-70d2-4f42-9329-78e5b7d44bcf None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:46.080794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-fa625d52-c57c-421a-93c9-2c1ddac4209b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:46.081784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fa625d52-c57c-421a-93c9-2c1ddac4209b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:46.118486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27c604-877f-4532-bc66-57a9e63363f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:46.119222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27c604-877f-4532-bc66-57a9e63363f0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.119510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27c604-877f-4532-bc66-57a9e63363f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.119786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27c604-877f-4532-bc66-57a9e63363f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.121021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27c604-877f-4532-bc66-57a9e63363f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:46.122526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27c604-877f-4532-bc66-57a9e63363f0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:46.126404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 505/1024] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:46.130565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27c604-877f-4532-bc66-57a9e63363f0 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:46.162011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 520/1025] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:46.205902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:46.232345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16722c8-6088-45e0-849b-65e4fdae97ab None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.232637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16722c8-6088-45e0-849b-65e4fdae97ab None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.232928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16722c8-6088-45e0-849b-65e4fdae97ab None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.234592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 521/1026] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:46.236545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27c604-877f-4532-bc66-57a9e63363f0 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:46.240351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233025ba-7612-4235-b4e5-a985f2f542ac None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.240636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233025ba-7612-4235-b4e5-a985f2f542ac None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.240908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233025ba-7612-4235-b4e5-a985f2f542ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.260847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-233025ba-7612-4235-b4e5-a985f2f542ac None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:46.262058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-233025ba-7612-4235-b4e5-a985f2f542ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:46.272113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57fa3-6679-4194-8fd8-13fb15a33dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:46.273634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57fa3-6679-4194-8fd8-13fb15a33dd6 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.273917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57fa3-6679-4194-8fd8-13fb15a33dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.274214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57fa3-6679-4194-8fd8-13fb15a33dd6 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.275193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa57fa3-6679-4194-8fd8-13fb15a33dd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:46.280433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa57fa3-6679-4194-8fd8-13fb15a33dd6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:46.346854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 522/1027] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:46.391007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 506/1028] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:34:46 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:34:46.440978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233025ba-7612-4235-b4e5-a985f2f542ac None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:46.444887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:46.559051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233025ba-7612-4235-b4e5-a985f2f542ac None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:46.597051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada7ce4-7327-4133-a462-e28a2a4084f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:46.597693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7ce4-7327-4133-a462-e28a2a4084f1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.597977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7ce4-7327-4133-a462-e28a2a4084f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.598406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7ce4-7327-4133-a462-e28a2a4084f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.610638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada7ce4-7327-4133-a462-e28a2a4084f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-5ada7ce4-7327-4133-a462-e28a2a4084f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:46.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:46.616689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:46.617553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:46.618236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57fa3-6679-4194-8fd8-13fb15a33dd6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:46.618647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 523/1029] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 13:34:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:46.624031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada7ce4-7327-4133-a462-e28a2a4084f1 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:46.699593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:46.737576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57fa3-6679-4194-8fd8-13fb15a33dd6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:46.739051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada7ce4-7327-4133-a462-e28a2a4084f1 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:46.765257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:46.773830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:46.774494 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.774770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.775039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] PATH_INFO: `/v3/services/image` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.775616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:46.777259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:46.777544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:46.778166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.778451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.778724 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.779388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:46.779920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:46.781038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:46.786530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:46.787541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:46.788592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 507/1030] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 8 13:34:46 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:46.797977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 524/1031] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 13:34:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:46.798166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:46.799624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:46.904930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1972b06-edaa-4741-99dc-403a43436135 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:46.905667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04308d-c328-4333-9bdc-ee19da21f39d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:46.933258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:46.942640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:46.945641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.945641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.945641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.947492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:46.948432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:46.948944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:46.949428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:46.950080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] RBAC: Adding query filter params (name=image) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:46.950409 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:46.951789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:46.957454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 508/1032] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:46.959882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:46.961386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:46.962993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:46.987255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:46.988282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:46.989280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:46.989280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:46.989280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:46.989280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 525/1033] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:46.994793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:47.032388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:47.078614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e48180-cd23-4db5-a54c-cfe4a8c07341 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:47.110020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550273-eac2-4509-a8da-998b92cdd9f2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:47.114605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:47.115224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:47.115504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:47.115776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:47.116402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] RBAC: Adding query filter params (type=image) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:47.117541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:47.118964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:47.126557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 509/1034] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:47.132328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:47.151964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:47.152593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:47.152855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:47.153124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:47.153768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:47.154088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:47.155538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:47.168047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 526/1035] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:47.173623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:47.216186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:47.253733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378064e-83c9-41a3-af19-77445f2e865b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:47.276741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:47.278514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87f5b2-b63e-47f8-8ae6-dff8ad691bd3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:47.291341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:47.292228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:47.292373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:47.292658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:47.293413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] RBAC: Adding query filter params (interface=public, service_id=7f7230694a3f4c6886e185f338350ed1, region_id=RegionOne) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:47.293726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:47.295127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:47.302310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 510/1036] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 13:34:47 2022] GET /identity/v3/endpoints?service_id=7f7230694a3f4c6886e185f338350ed1&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:47.307482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:47.321745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:47.322643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:47.323073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:47.340573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:47.340573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:47.340573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:47.345057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:47.346238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:47.347427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:47.349284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 527/1037] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:34:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:47.355330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:47.416477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd278f-b376-421f-93f4-5e263e7a29ca None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:47.452920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ef42d-c2b8-4212-bc89-58c55cab0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:47.453760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ef42d-c2b8-4212-bc89-58c55cab0189 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:47.454045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ef42d-c2b8-4212-bc89-58c55cab0189 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:47.454347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ef42d-c2b8-4212-bc89-58c55cab0189 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:47.454952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ef42d-c2b8-4212-bc89-58c55cab0189 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:47.456307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ef42d-c2b8-4212-bc89-58c55cab0189 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:47.459149 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:47.463151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915ae1-2915-49b6-a8a6-8bf97cbce232 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:47.463273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 511/1038] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:34:47 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:34:47.501526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:47.502201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:47.502530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:47.502805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:47.503581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:47.503889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:47.505364 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:47.530182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 528/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:34:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:47.531483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ef42d-c2b8-4212-bc89-58c55cab0189 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:47.586902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:47.651684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ef42d-c2b8-4212-bc89-58c55cab0189 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:47.687964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b615b75-feb3-41f0-ba06-06b4eb98445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:47.688800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b615b75-feb3-41f0-ba06-06b4eb98445e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:47.689241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b615b75-feb3-41f0-ba06-06b4eb98445e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:47.689653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b615b75-feb3-41f0-ba06-06b4eb98445e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:47.760770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b615b75-feb3-41f0-ba06-06b4eb98445e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:47.762475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b615b75-feb3-41f0-ba06-06b4eb98445e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:47.810554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:47.863167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 512/1040] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:47 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:47.869701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:47.971222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b80fde-e939-4d7d-8a2a-d131838803f3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:48.005723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:48.006045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:48.006332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:48.006606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:48.017403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:48.018888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:48.033563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:48.034572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:48.035489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:48.035489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:48.035489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:48.039029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 529/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:48.041540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b615b75-feb3-41f0-ba06-06b4eb98445e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:48.141440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b615b75-feb3-41f0-ba06-06b4eb98445e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:48.185317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:48.185966 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:48.186270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:48.186546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:48.187199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:48.187502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:48.188858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:48.202189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 513/1042] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:48.207383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:48.228445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:48.325260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cbbf7-d71a-4fb2-92d2-f54c4cc084d2 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:48.363136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:48.363973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:48.364349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:48.364932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:48.373031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:48.374914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:48.387230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:48.388358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:48.389578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:48.389578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:48.389578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:48.389578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:48.389578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:48.389578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:48.389578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:48.389578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:48.397040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 530/1043] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:48.397190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:48.413558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:48.533929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609860b6-9da7-44d8-b88b-5e37e5ae683d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:48.577256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4ab98-3a38-415f-a4e7-9e3eaf3b8a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:48.578187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ab98-3a38-415f-a4e7-9e3eaf3b8a07 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:48.578641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ab98-3a38-415f-a4e7-9e3eaf3b8a07 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:48.579077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ab98-3a38-415f-a4e7-9e3eaf3b8a07 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:48.579953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4ab98-3a38-415f-a4e7-9e3eaf3b8a07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:48.580422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4ab98-3a38-415f-a4e7-9e3eaf3b8a07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:48.582004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4ab98-3a38-415f-a4e7-9e3eaf3b8a07 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:48.597275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 514/1044] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:48.607123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:48.669708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7b427-29fa-4064-ad5f-90415055efbc None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:48.670267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7b427-29fa-4064-ad5f-90415055efbc None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:48.670520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7b427-29fa-4064-ad5f-90415055efbc None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:48.672007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 515/1045] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:48.677297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98552-81fc-4531-9252-5807c14d0c89 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:48.677607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98552-81fc-4531-9252-5807c14d0c89 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:48.677889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98552-81fc-4531-9252-5807c14d0c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:48.683115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:48.696335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-18b98552-81fc-4531-9252-5807c14d0c89 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:48.697539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-18b98552-81fc-4531-9252-5807c14d0c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:48.721856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c865630-bd24-425c-ac3f-c0a17fe09704 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:48.759972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:48.760581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:48.760861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:48.761137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:48.762042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:48.762902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] RBAC: Adding query filter params (role.id=1ab14f52bb114cefadb70cd2a83acf19, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=c24d70e571c64f018aa199c4c08d153e) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:48.763334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:48.764779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:48.767725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 516/1046] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:48.782164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 531/1047] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:48 2022] GET /identity/v3/role_assignments?user.id=c24d70e571c64f018aa199c4c08d153e&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=1ab14f52bb114cefadb70cd2a83acf19 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:49.045069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b98552-81fc-4531-9252-5807c14d0c89 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:49.145863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b98552-81fc-4531-9252-5807c14d0c89 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:49.182934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4236bf-4efa-4bef-815c-8c29c1cb879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:49.183642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4236bf-4efa-4bef-815c-8c29c1cb879a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:49.183933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4236bf-4efa-4bef-815c-8c29c1cb879a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:49.184211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4236bf-4efa-4bef-815c-8c29c1cb879a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:49.194812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4236bf-4efa-4bef-815c-8c29c1cb879a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-4c4236bf-4efa-4bef-815c-8c29c1cb879a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:49.199682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:49.200618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:49.201482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:49.206196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 517/1048] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:49.206944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:49.313253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82057f2-fa61-4f4c-9c79-6ba560ea608e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:49.370395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:49.375887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:49.376794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:49.377395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:49.379168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:49.379168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:49.379168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:49.380541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:49.391571 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 532/1049] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:49.397631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4236bf-4efa-4bef-815c-8c29c1cb879a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:49.459084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:49.503804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4236bf-4efa-4bef-815c-8c29c1cb879a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:49.544040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c35bce-3806-45c2-911d-dd7770b6890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:49.544705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35bce-3806-45c2-911d-dd7770b6890a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:49.544986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35bce-3806-45c2-911d-dd7770b6890a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:49.545261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35bce-3806-45c2-911d-dd7770b6890a None admin] PATH_INFO: `/v3/users/21f1d1fcd3d74912aaf4afe12a12d043` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:49.561352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c35bce-3806-45c2-911d-dd7770b6890a None admin] RBAC: Authorizing `identity:get_user(user_id=21f1d1fcd3d74912aaf4afe12a12d043)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:49.562906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c35bce-3806-45c2-911d-dd7770b6890a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:49.569327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 518/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:34:49 2022] GET /identity/v3/users/21f1d1fcd3d74912aaf4afe12a12d043 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:49.571216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:49.656916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6277-27ba-4048-aff6-138a8b71f218 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:49.657204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6277-27ba-4048-aff6-138a8b71f218 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:49.657486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6277-27ba-4048-aff6-138a8b71f218 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:49.658999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 519/1051] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:49.664683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955dd86-2094-470a-8559-d2012904245c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:49.664957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955dd86-2094-470a-8559-d2012904245c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:49.665227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955dd86-2094-470a-8559-d2012904245c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:49.684535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6955dd86-2094-470a-8559-d2012904245c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:49.685762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6955dd86-2094-470a-8559-d2012904245c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:49.705074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfbbef-d997-4be6-8de2-e71200b8478b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:49.743713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ba181-6913-43de-b4e0-ea92f7053951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:49.744327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ba181-6913-43de-b4e0-ea92f7053951 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:49.744608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ba181-6913-43de-b4e0-ea92f7053951 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:49.744880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ba181-6913-43de-b4e0-ea92f7053951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:49.765610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 520/1052] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:49.812285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:49.820513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ba181-6913-43de-b4e0-ea92f7053951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:49.821894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ba181-6913-43de-b4e0-ea92f7053951 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:49.926302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 533/1053] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:49 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:49.927703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955dd86-2094-470a-8559-d2012904245c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:49.975143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:50.028798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955dd86-2094-470a-8559-d2012904245c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:50.037678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ba181-6913-43de-b4e0-ea92f7053951 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:50.069113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ee49c-7fbd-464c-ac41-04e0e9de8b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:50.069802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ee49c-7fbd-464c-ac41-04e0e9de8b94 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:50.070088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ee49c-7fbd-464c-ac41-04e0e9de8b94 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:50.070395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ee49c-7fbd-464c-ac41-04e0e9de8b94 None admin] PATH_INFO: `/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:50.071621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ee49c-7fbd-464c-ac41-04e0e9de8b94 None admin] RBAC: Authorizing `identity:get_project(project_id=480a1b8eb4974b33b6a52e5d5c903e08)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:50.079505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ee49c-7fbd-464c-ac41-04e0e9de8b94 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:50.088514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 521/1054] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:34:49 2022] GET /identity/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:50.089943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ee49c-7fbd-464c-ac41-04e0e9de8b94 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:50.194655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ee49c-7fbd-464c-ac41-04e0e9de8b94 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:50.202313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:50.208779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ba181-6913-43de-b4e0-ea92f7053951 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:50.215617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:50.235802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:50.236731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:50.237008 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:50.237292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:50.238001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=480a1b8eb4974b33b6a52e5d5c903e08, user.id=21f1d1fcd3d74912aaf4afe12a12d043) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:50.238345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:50.239823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:50.245146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:50.245283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 522/1055] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:50 2022] GET /identity/v3/role_assignments?user.id=21f1d1fcd3d74912aaf4afe12a12d043&scope.project.id=480a1b8eb4974b33b6a52e5d5c903e08&role.id=97f11873550e41dba0cd095afb1dec00 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:50.245776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:50.246052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:50.246353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] PATH_INFO: `/v3/services/image` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:50.246933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:50.248401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:50.258480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:50.259574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:34:50.260524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:50.267166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 534/1056] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 8 13:34:50 2022] GET /identity/v3/services/image => generated 86 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:50.267242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:50.379127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b24e0-72ba-4404-aad9-d445fea77615 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:50.417490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:50.418125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:50.418431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:50.418740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:50.419356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] RBAC: Adding query filter params (name=image) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:50.419654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:50.421037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:50.429435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 523/1057] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:50.439083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:50.552943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720acf16-c06e-4f0f-81b4-9d55b7c2bccd None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:50.587521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84538582-9b17-4436-9607-f56844c901cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:50.588127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84538582-9b17-4436-9607-f56844c901cd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:50.588404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84538582-9b17-4436-9607-f56844c901cd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:50.588675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84538582-9b17-4436-9607-f56844c901cd None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:50.589302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84538582-9b17-4436-9607-f56844c901cd None admin] RBAC: Adding query filter params (type=image) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:50.589867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84538582-9b17-4436-9607-f56844c901cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:50.593230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84538582-9b17-4436-9607-f56844c901cd None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:50.605164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 535/1058] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:50.606376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:50.655241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:50.713147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede4fd8-df99-438a-81d4-49cc5e41242e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:50.749668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656584a1-0d62-4de9-b1e1-a00f5fca3e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:50.750318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656584a1-0d62-4de9-b1e1-a00f5fca3e4a None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:50.750604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656584a1-0d62-4de9-b1e1-a00f5fca3e4a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:50.750880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656584a1-0d62-4de9-b1e1-a00f5fca3e4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:50.751552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656584a1-0d62-4de9-b1e1-a00f5fca3e4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:50.752922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656584a1-0d62-4de9-b1e1-a00f5fca3e4a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:50.856284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 524/1059] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:34:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:34:51.008289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a90e29-6c6e-47ab-8fa9-3dc2178def9a None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:51.008488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a90e29-6c6e-47ab-8fa9-3dc2178def9a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:51.008759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a90e29-6c6e-47ab-8fa9-3dc2178def9a None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:51.012791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 536/1060] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:51.016588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4db4b6-6d97-4a51-a273-35f86486d1ff None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:51.016831 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4db4b6-6d97-4a51-a273-35f86486d1ff None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:51.017044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4db4b6-6d97-4a51-a273-35f86486d1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:51.037607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-fd4db4b6-6d97-4a51-a273-35f86486d1ff None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:51.039406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fd4db4b6-6d97-4a51-a273-35f86486d1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:51.124091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 525/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:51.392770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a90e29-6c6e-47ab-8fa9-3dc2178def9a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:51.419212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:51.500888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a90e29-6c6e-47ab-8fa9-3dc2178def9a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:51.535377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3fb90-47a9-4156-99d7-bf119de275b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:51.536016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fb90-47a9-4156-99d7-bf119de275b3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:51.536668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fb90-47a9-4156-99d7-bf119de275b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:51.536668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fb90-47a9-4156-99d7-bf119de275b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:51.548835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3fb90-47a9-4156-99d7-bf119de275b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-0ce3fb90-47a9-4156-99d7-bf119de275b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:51.554123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:51.555169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:51.556111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:51.558212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 537/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 13:34:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:51.561720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4db4b6-6d97-4a51-a273-35f86486d1ff None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:51.588146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:51.678747 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4db4b6-6d97-4a51-a273-35f86486d1ff None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:51.713893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:51.714812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:51.715086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:51.715340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:51.716006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:51.716317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:51.717682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:51.731438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 526/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 13:34:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:51.732780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3fb90-47a9-4156-99d7-bf119de275b3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:51.823457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:51.843117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3fb90-47a9-4156-99d7-bf119de275b3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:51.879645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:51.880286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:51.880569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:51.880846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:51.892134 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:51.893580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:51.908388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:51.909423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:51.910838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:51.910838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:51.910838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:51.914486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 538/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:51.915709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:52.002802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:52.018488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef9d05-32b6-476a-9cae-e4d3f543cb31 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:52.052099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:52.052752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.053029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.053353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:52.054022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:52.054348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:52.055700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:52.072761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 527/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:52.074230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:52.174828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062d14-7ab4-4cf5-a1f7-f7ef2d35ee23 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:52.209438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:52.210094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.210395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.210665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:52.212517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:52.219261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:52.220691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:52.231964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:52.233084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:52.234049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 539/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:34:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:52.238824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:52.341593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255721f8-f616-462e-ab45-42f8030c647b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:52.377954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1dbda-ee3d-431c-bd5f-661bc2198905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:52.378597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1dbda-ee3d-431c-bd5f-661bc2198905 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.378879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1dbda-ee3d-431c-bd5f-661bc2198905 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.379151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1dbda-ee3d-431c-bd5f-661bc2198905 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:52.379880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1dbda-ee3d-431c-bd5f-661bc2198905 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:52.380182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1dbda-ee3d-431c-bd5f-661bc2198905 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:52.381600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1dbda-ee3d-431c-bd5f-661bc2198905 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:52.399656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 528/1067] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:34:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:52.405196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:52.449669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f964c-52ef-4494-8dc0-13ebac620812 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.450000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f964c-52ef-4494-8dc0-13ebac620812 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.450243 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f964c-52ef-4494-8dc0-13ebac620812 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:52.451583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 529/1068] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:52.456637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733a7a2-264b-4e90-977e-d9f436f0206c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.456930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733a7a2-264b-4e90-977e-d9f436f0206c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.457212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733a7a2-264b-4e90-977e-d9f436f0206c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:52.464840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:52.476518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2733a7a2-264b-4e90-977e-d9f436f0206c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:52.478350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2733a7a2-264b-4e90-977e-d9f436f0206c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:52.519912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2260d2-4d2f-42cd-b4da-c7b8e1360b0a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:52.556240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:52.556880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.557162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.557437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:52.561133 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 530/1069] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:34:52.568395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:52.569887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:52.584373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:52.585489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:52.591709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:52.591709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:52.591709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:52.591709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 540/1070] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:52.593271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733a7a2-264b-4e90-977e-d9f436f0206c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:52.698614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733a7a2-264b-4e90-977e-d9f436f0206c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:52.733355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:52.734013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.734275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.734518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:52.735156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:52.735464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:52.736817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:52.738374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:52.749252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 531/1071] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:52.754673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:52.802275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:52.835822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:52.863119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f6c0-2db0-4bc3-8db1-5156a1c45df0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:52.906321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:52.907269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.907726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.908018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:52.915004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:52.916462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:52.929328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:52.930342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:52.931251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:52.931251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:52.931251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:52.931251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:52.931251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:52.931251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:52.931251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:52.931251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:52.935342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 541/1072] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:52.936069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:52.953992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767049f-44b8-444f-b8ea-d71157a82905 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:52.996245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a1a8f-6c91-4131-ab6b-a75252bc3055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:52.996881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a1a8f-6c91-4131-ab6b-a75252bc3055 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:52.997167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a1a8f-6c91-4131-ab6b-a75252bc3055 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:52.997442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a1a8f-6c91-4131-ab6b-a75252bc3055 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.009566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a1a8f-6c91-4131-ab6b-a75252bc3055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-d27a1a8f-6c91-4131-ab6b-a75252bc3055 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:53.014416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:53.015350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:53.016207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 532/1073] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:53.021829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a1a8f-6c91-4131-ab6b-a75252bc3055 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:53.047895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477a9da-e1a9-42c5-9e79-b4c5abe930ae None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:53.056365 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:53.082238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:53.082428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fddce-5819-4815-ae29-93ec246e98f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:53.083124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fddce-5819-4815-ae29-93ec246e98f1 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.083124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fddce-5819-4815-ae29-93ec246e98f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.083461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fddce-5819-4815-ae29-93ec246e98f1 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.084075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fddce-5819-4815-ae29-93ec246e98f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:53.084376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fddce-5819-4815-ae29-93ec246e98f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:53.086351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fddce-5819-4815-ae29-93ec246e98f1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:53.099099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 542/1074] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:53.100291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925c4b7-0025-45d2-b787-f1bdeddf49be None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.101140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925c4b7-0025-45d2-b787-f1bdeddf49be None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.101413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925c4b7-0025-45d2-b787-f1bdeddf49be None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.104883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 543/1075] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:53.112152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925c4b7-0025-45d2-b787-f1bdeddf49be None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:53.127702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a1a8f-6c91-4131-ab6b-a75252bc3055 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:53.173527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306bdc5-2cd6-4d46-be5f-62ab9da97558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:53.174181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306bdc5-2cd6-4d46-be5f-62ab9da97558 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.174464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306bdc5-2cd6-4d46-be5f-62ab9da97558 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.174742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306bdc5-2cd6-4d46-be5f-62ab9da97558 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.175419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306bdc5-2cd6-4d46-be5f-62ab9da97558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:53.175727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306bdc5-2cd6-4d46-be5f-62ab9da97558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:53.177558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306bdc5-2cd6-4d46-be5f-62ab9da97558 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:53.197293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 533/1076] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:53.198668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f509-7795-42a9-a294-3832e039dda6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.199067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f509-7795-42a9-a294-3832e039dda6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.199227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f509-7795-42a9-a294-3832e039dda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.217376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9823f509-7795-42a9-a294-3832e039dda6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:53.218598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9823f509-7795-42a9-a294-3832e039dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:53.223470 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925c4b7-0025-45d2-b787-f1bdeddf49be None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:53.258004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f4080-4706-4531-8d9f-f9f6586ebe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:53.258690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f4080-4706-4531-8d9f-f9f6586ebe9e None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.258966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f4080-4706-4531-8d9f-f9f6586ebe9e None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.259235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f4080-4706-4531-8d9f-f9f6586ebe9e None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/c24d70e571c64f018aa199c4c08d153e/roles/1ab14f52bb114cefadb70cd2a83acf19` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.261416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f4080-4706-4531-8d9f-f9f6586ebe9e None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=c24d70e571c64f018aa199c4c08d153e, role_id=1ab14f52bb114cefadb70cd2a83acf19)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:53.262944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f4080-4706-4531-8d9f-f9f6586ebe9e None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:53.284311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 534/1077] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:53.285234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823f509-7795-42a9-a294-3832e039dda6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:53.330498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:53.373869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 544/1078] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:34:53 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/c24d70e571c64f018aa199c4c08d153e/roles/1ab14f52bb114cefadb70cd2a83acf19 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:34:53.392554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823f509-7795-42a9-a294-3832e039dda6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:53.432135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce77ce-c31d-43bf-9d71-7d72f5ca50fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:53.432804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce77ce-c31d-43bf-9d71-7d72f5ca50fb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.433085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce77ce-c31d-43bf-9d71-7d72f5ca50fb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.433362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce77ce-c31d-43bf-9d71-7d72f5ca50fb None admin] PATH_INFO: `/v3/users/21f1d1fcd3d74912aaf4afe12a12d043` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.434640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce77ce-c31d-43bf-9d71-7d72f5ca50fb None admin] RBAC: Authorizing `identity:get_user(user_id=21f1d1fcd3d74912aaf4afe12a12d043)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:53.436120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce77ce-c31d-43bf-9d71-7d72f5ca50fb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:53.437993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 535/1079] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:34:53 2022] GET /identity/v3/users/21f1d1fcd3d74912aaf4afe12a12d043 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:53.443958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f4080-4706-4531-8d9f-f9f6586ebe9e None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:53.473919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:53.568256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f4080-4706-4531-8d9f-f9f6586ebe9e None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:53.570036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce77ce-c31d-43bf-9d71-7d72f5ca50fb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:53.610503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d695d-5991-43b4-b736-169421e35b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:53.611138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d695d-5991-43b4-b736-169421e35b1d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.611413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d695d-5991-43b4-b736-169421e35b1d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.611683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d695d-5991-43b4-b736-169421e35b1d None admin] PATH_INFO: `/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.612799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d695d-5991-43b4-b736-169421e35b1d None admin] RBAC: Authorizing `identity:get_project(project_id=480a1b8eb4974b33b6a52e5d5c903e08)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:53.614727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d695d-5991-43b4-b736-169421e35b1d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:53.618254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 545/1080] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:34:53 2022] GET /identity/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:53.623799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d695d-5991-43b4-b736-169421e35b1d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:53.691873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce77ce-c31d-43bf-9d71-7d72f5ca50fb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:53.711224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:53.733437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:53.736413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0ad3a-f334-4e7e-982c-776a8b6723ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:53.737195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0ad3a-f334-4e7e-982c-776a8b6723ae None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.737791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0ad3a-f334-4e7e-982c-776a8b6723ae None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.738066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0ad3a-f334-4e7e-982c-776a8b6723ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.740337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d695d-5991-43b4-b736-169421e35b1d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:53.775469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b3ec-abf7-479e-b9e0-c11b66a66ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:53.776133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b3ec-abf7-479e-b9e0-c11b66a66ed8 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:53.776412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b3ec-abf7-479e-b9e0-c11b66a66ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:53.776688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b3ec-abf7-479e-b9e0-c11b66a66ed8 None admin] PATH_INFO: `/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08/users/21f1d1fcd3d74912aaf4afe12a12d043/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:53.778989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771b3ec-abf7-479e-b9e0-c11b66a66ed8 None admin] RBAC: Authorizing `identity:create_grant(project_id=480a1b8eb4974b33b6a52e5d5c903e08, user_id=21f1d1fcd3d74912aaf4afe12a12d043, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:53.780885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771b3ec-abf7-479e-b9e0-c11b66a66ed8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:53.810613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0ad3a-f334-4e7e-982c-776a8b6723ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:53.812089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0ad3a-f334-4e7e-982c-776a8b6723ae None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:53.842722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 546/1081] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:34:53 2022] PUT /identity/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08/users/21f1d1fcd3d74912aaf4afe12a12d043/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:34:53.922519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 536/1082] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:34:53.927383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b3ec-abf7-479e-b9e0-c11b66a66ed8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:54.041628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b3ec-abf7-479e-b9e0-c11b66a66ed8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:54.078607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa18a87-5e22-4d4e-9216-b063c0c35922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:54.079259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa18a87-5e22-4d4e-9216-b063c0c35922 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:54.079537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa18a87-5e22-4d4e-9216-b063c0c35922 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:54.079805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa18a87-5e22-4d4e-9216-b063c0c35922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:54.090504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa18a87-5e22-4d4e-9216-b063c0c35922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:54.091960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa18a87-5e22-4d4e-9216-b063c0c35922 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-eaa18a87-5e22-4d4e-9216-b063c0c35922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:54.109946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:54.111156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:54.112141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:54.112141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:54.112141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:54.112141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 547/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:54.119183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0ad3a-f334-4e7e-982c-776a8b6723ae None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:54.135542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:54.220206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0ad3a-f334-4e7e-982c-776a8b6723ae None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:54.254045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199f968-5407-4097-9f68-3dbc60c4564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:54.256187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199f968-5407-4097-9f68-3dbc60c4564c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:54.256187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199f968-5407-4097-9f68-3dbc60c4564c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:54.256187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199f968-5407-4097-9f68-3dbc60c4564c None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:54.256187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199f968-5407-4097-9f68-3dbc60c4564c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:54.256187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199f968-5407-4097-9f68-3dbc60c4564c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:54.257474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199f968-5407-4097-9f68-3dbc60c4564c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:54.274169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 537/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:55.585253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea70e-8354-405e-9688-fcd67ead03ae None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:55.585821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea70e-8354-405e-9688-fcd67ead03ae None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:55.585821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea70e-8354-405e-9688-fcd67ead03ae None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:55.587252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 548/1085] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:55.594813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e900900-5e69-4e71-bfb9-3b98f4f3c92a None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:55.594813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e900900-5e69-4e71-bfb9-3b98f4f3c92a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:55.594813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e900900-5e69-4e71-bfb9-3b98f4f3c92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:55.612218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-7e900900-5e69-4e71-bfb9-3b98f4f3c92a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:55.613405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7e900900-5e69-4e71-bfb9-3b98f4f3c92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:55.688612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 538/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:55.956729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eea70e-8354-405e-9688-fcd67ead03ae None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:55.966262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:56.016392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455048d9-926e-40a2-98ba-650875424cd1 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.016708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455048d9-926e-40a2-98ba-650875424cd1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.016955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455048d9-926e-40a2-98ba-650875424cd1 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.018469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 539/1087] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:34:56.023296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d60e-5bd3-45c9-ac4c-1cf4149a6f85 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.023574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d60e-5bd3-45c9-ac4c-1cf4149a6f85 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.023847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d60e-5bd3-45c9-ac4c-1cf4149a6f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.030824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:56.042247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c3b2d60e-5bd3-45c9-ac4c-1cf4149a6f85 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:56.043639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c3b2d60e-5bd3-45c9-ac4c-1cf4149a6f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:56.060677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eea70e-8354-405e-9688-fcd67ead03ae None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.094687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18e3b7-d252-4309-9e5f-5411c549d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.095312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18e3b7-d252-4309-9e5f-5411c549d96b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.095593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18e3b7-d252-4309-9e5f-5411c549d96b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.095868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18e3b7-d252-4309-9e5f-5411c549d96b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.106415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18e3b7-d252-4309-9e5f-5411c549d96b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:56.110420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 540/1088] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-aa18e3b7-d252-4309-9e5f-5411c549d96b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:56.111211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:56.112378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:56.113392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 549/1089] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 13:34:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:56.118746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d60e-5bd3-45c9-ac4c-1cf4149a6f85 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.214185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d60e-5bd3-45c9-ac4c-1cf4149a6f85 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.247416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.248021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.254253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.254253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.254253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:56.254253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:56.254253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:56.254253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:56.259833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 541/1090] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:56.265801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18e3b7-d252-4309-9e5f-5411c549d96b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.330984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:56.365225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18e3b7-d252-4309-9e5f-5411c549d96b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.381955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.398985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.399618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.399912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.400178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.410793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:56.412239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:56.426474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:56.427621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:56.429441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:56.429441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:56.429441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:56.429441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 550/1091] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:34:56.433749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.485582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356f940-e7a3-48ac-86da-8ec7b37f5b87 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.518031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4673d4c-9cae-4db8-a3c3-a0627fcae6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.518688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4673d4c-9cae-4db8-a3c3-a0627fcae6ae None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.518961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4673d4c-9cae-4db8-a3c3-a0627fcae6ae None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.519224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4673d4c-9cae-4db8-a3c3-a0627fcae6ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.529467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4673d4c-9cae-4db8-a3c3-a0627fcae6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b4673d4c-9cae-4db8-a3c3-a0627fcae6ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:34:56.534216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:56.535208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:34:56.536077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:56.536808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1466409-60f6-4865-b0a6-1a45fd29cb7a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.536886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 542/1092] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:56.541847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4673d4c-9cae-4db8-a3c3-a0627fcae6ae None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.554473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:56.571082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.571687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.571966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.572237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.572874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:56.573229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:56.574675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:56.587119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 551/1093] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:56.593140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.601719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:56.651100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4673d4c-9cae-4db8-a3c3-a0627fcae6ae None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.684872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.685477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.685760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.686033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.686729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:56.687035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:56.688404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:56.694510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef9379-49fc-4a54-abbf-4af2b51a29a4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.696273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 543/1094] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:56.702178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.729657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.730320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.730605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.730883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.737999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:56.739431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:34:56.752157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:56.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:56.754233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 552/1095] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:56.759231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.800109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:56.805030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d015c7-b3c6-4d1e-8826-f567ffbc54ba None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.822495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:56.839122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7cdca-14b6-47ec-89e7-6aaea01902a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.840011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7cdca-14b6-47ec-89e7-6aaea01902a6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.840289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7cdca-14b6-47ec-89e7-6aaea01902a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.840561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7cdca-14b6-47ec-89e7-6aaea01902a6 None admin] PATH_INFO: `/v3/users/21f1d1fcd3d74912aaf4afe12a12d043` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.841696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7cdca-14b6-47ec-89e7-6aaea01902a6 None admin] RBAC: Authorizing `identity:get_user(user_id=21f1d1fcd3d74912aaf4afe12a12d043)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:56.843162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7cdca-14b6-47ec-89e7-6aaea01902a6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:56.844738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 544/1096] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/users/21f1d1fcd3d74912aaf4afe12a12d043 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:56.850996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7cdca-14b6-47ec-89e7-6aaea01902a6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.861963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96c3b-8bce-412d-86f1-e2832d98dd96 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.894742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.895350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.895628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.895900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:56.896643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:56.896949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:56.898928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:56.916291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 553/1097] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:56.921986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:56.950330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7cdca-14b6-47ec-89e7-6aaea01902a6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:56.985748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54dcee-f1a1-4204-adbb-0092dc3ecd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:56.987924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54dcee-f1a1-4204-adbb-0092dc3ecd2a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:56.988264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54dcee-f1a1-4204-adbb-0092dc3ecd2a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:56.988491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54dcee-f1a1-4204-adbb-0092dc3ecd2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:57.014166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:57.019843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9202bd-5f5c-4559-8ad2-cdc47dd9d69c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:57.045374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:57.054417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a3171-636a-4d03-a3d2-65878dfa1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:57.055024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a3171-636a-4d03-a3d2-65878dfa1a2d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:57.055301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a3171-636a-4d03-a3d2-65878dfa1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:57.055612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a3171-636a-4d03-a3d2-65878dfa1a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:57.058279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54dcee-f1a1-4204-adbb-0092dc3ecd2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:57.059782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54dcee-f1a1-4204-adbb-0092dc3ecd2a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:57.121252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a3171-636a-4d03-a3d2-65878dfa1a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:57.122728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a3171-636a-4d03-a3d2-65878dfa1a2d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:57.158833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 545/1098] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:34:57.164843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54dcee-f1a1-4204-adbb-0092dc3ecd2a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:57.221325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:57.228064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 554/1099] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:34:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:34:57.233724 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a3171-636a-4d03-a3d2-65878dfa1a2d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:57.252117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:57.267447 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54dcee-f1a1-4204-adbb-0092dc3ecd2a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:57.305198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc197070-7ecc-4179-9f23-9a15d27475e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:57.305825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc197070-7ecc-4179-9f23-9a15d27475e0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:57.310644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc197070-7ecc-4179-9f23-9a15d27475e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:57.310644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc197070-7ecc-4179-9f23-9a15d27475e0 None admin] PATH_INFO: `/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:57.310644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc197070-7ecc-4179-9f23-9a15d27475e0 None admin] RBAC: Authorizing `identity:get_project(project_id=480a1b8eb4974b33b6a52e5d5c903e08)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:57.310644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc197070-7ecc-4179-9f23-9a15d27475e0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:57.310644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 546/1100] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:34:57 2022] GET /identity/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:57.316740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc197070-7ecc-4179-9f23-9a15d27475e0 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:57.336469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a3171-636a-4d03-a3d2-65878dfa1a2d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:57.371296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:57.371922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:57.372199 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:57.372471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:57.382463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:57.383903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:34:57.398278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:57.399273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:34:57.400638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:34:57.400638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:34:57.400638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:34:57.400638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 555/1101] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:57.404944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:57.419289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc197070-7ecc-4179-9f23-9a15d27475e0 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:57.453160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:57.453767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:57.454053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:57.454298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:57.454421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:57.455060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=480a1b8eb4974b33b6a52e5d5c903e08, user.id=21f1d1fcd3d74912aaf4afe12a12d043) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:57.455358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:57.456768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:57.463212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 547/1102] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:57 2022] GET /identity/v3/role_assignments?user.id=21f1d1fcd3d74912aaf4afe12a12d043&scope.project.id=480a1b8eb4974b33b6a52e5d5c903e08&role.id=97f11873550e41dba0cd095afb1dec00 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:57.509781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e533e-f2ca-4377-af97-257add2244c4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:57.546633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:57.547247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:57.547523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:57.547793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:57.548445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:57.548746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:57.550124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:57.562988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 556/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:57.569444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:57.592360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:57.679794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060dbb1-62c6-407d-828f-9c8e55075b55 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:57.718595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:57.719226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:57.719506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:57.719777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:57.727552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:57.728981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:34:57.740193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:34:57.741302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:34:57.742289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 548/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:34:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:34:57.747310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:57.846332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0181c3-51c2-4d58-9f66-dff2c6fbb031 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:57.855932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:57.879517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceaeb4-5768-48cf-8ac2-05d0022e216a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:57.880121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceaeb4-5768-48cf-8ac2-05d0022e216a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:57.880394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceaeb4-5768-48cf-8ac2-05d0022e216a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:57.880662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceaeb4-5768-48cf-8ac2-05d0022e216a None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:57.881954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cceaeb4-5768-48cf-8ac2-05d0022e216a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:57.882308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cceaeb4-5768-48cf-8ac2-05d0022e216a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:57.883717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cceaeb4-5768-48cf-8ac2-05d0022e216a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:57.895381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 557/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:34:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:34:57.902050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:34:57.978530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:34:57.999813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a964e-bd9b-4c7f-923e-bd8b5d38c0e9 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:34:58.033451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7660a1-8b2b-4281-8326-27707ab8bc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:34:58.034060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7660a1-8b2b-4281-8326-27707ab8bc11 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:58.034369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7660a1-8b2b-4281-8326-27707ab8bc11 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:58.034643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7660a1-8b2b-4281-8326-27707ab8bc11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:58.035373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7660a1-8b2b-4281-8326-27707ab8bc11 None admin] RBAC: Adding query filter params (role.id=1ab14f52bb114cefadb70cd2a83acf19, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=c24d70e571c64f018aa199c4c08d153e) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:34:58.035702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7660a1-8b2b-4281-8326-27707ab8bc11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:34:58.037126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7660a1-8b2b-4281-8326-27707ab8bc11 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:34:58.042866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 549/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:34:57 2022] GET /identity/v3/role_assignments?user.id=c24d70e571c64f018aa199c4c08d153e&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=1ab14f52bb114cefadb70cd2a83acf19 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:34:59.670672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d3435-16c8-41c7-adbb-72504697b4f3 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:59.671125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d3435-16c8-41c7-adbb-72504697b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:59.671214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d3435-16c8-41c7-adbb-72504697b4f3 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:59.672569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 558/1107] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:34:59.677883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7c270-5fa1-4a9c-afb7-150a6d467ea2 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:34:59.679028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7c270-5fa1-4a9c-afb7-150a6d467ea2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:34:59.679169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7c270-5fa1-4a9c-afb7-150a6d467ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:34:59.699240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-edf7c270-5fa1-4a9c-afb7-150a6d467ea2 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:34:59.700432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-edf7c270-5fa1-4a9c-afb7-150a6d467ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:34:59.764868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 550/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:34:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:35:00.029089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d3435-16c8-41c7-adbb-72504697b4f3 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:00.133435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d3435-16c8-41c7-adbb-72504697b4f3 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:00.168048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad84313-6a72-4867-83b4-8ff6138b9516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:00.168693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84313-6a72-4867-83b4-8ff6138b9516 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:00.168972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84313-6a72-4867-83b4-8ff6138b9516 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:00.169243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84313-6a72-4867-83b4-8ff6138b9516 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:00.170402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad84313-6a72-4867-83b4-8ff6138b9516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:00.171842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad84313-6a72-4867-83b4-8ff6138b9516 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:00.173422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 559/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:35:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:00.179366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7c270-5fa1-4a9c-afb7-150a6d467ea2 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:00.249022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:00.281428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7c270-5fa1-4a9c-afb7-150a6d467ea2 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:00.320751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c38a52-d824-4eb9-9ec1-a9039bf8d434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:00.321377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c38a52-d824-4eb9-9ec1-a9039bf8d434 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:00.321658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c38a52-d824-4eb9-9ec1-a9039bf8d434 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:00.321931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c38a52-d824-4eb9-9ec1-a9039bf8d434 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:00.322630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c38a52-d824-4eb9-9ec1-a9039bf8d434 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:00.324115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c38a52-d824-4eb9-9ec1-a9039bf8d434 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:00.430992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 551/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:35:00 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:35:02.742480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff5c7f-7beb-498c-b98b-dabbd6975901 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:02.744411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff5c7f-7beb-498c-b98b-dabbd6975901 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:02.744411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff5c7f-7beb-498c-b98b-dabbd6975901 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:02.744411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 560/1111] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:02.750244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ada8b-84d2-42f1-b32a-21b66d0594c6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:02.750666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ada8b-84d2-42f1-b32a-21b66d0594c6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:02.750802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ada8b-84d2-42f1-b32a-21b66d0594c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:02.769024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ef1ada8b-84d2-42f1-b32a-21b66d0594c6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:02.770268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ef1ada8b-84d2-42f1-b32a-21b66d0594c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:02.839687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 552/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:03.108306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff5c7f-7beb-498c-b98b-dabbd6975901 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:03.144467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:03.218218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff5c7f-7beb-498c-b98b-dabbd6975901 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:03.255040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c3392-f15c-48e7-82d5-83a081d6032a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:03.255675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c3392-f15c-48e7-82d5-83a081d6032a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:03.256465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c3392-f15c-48e7-82d5-83a081d6032a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:03.256746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c3392-f15c-48e7-82d5-83a081d6032a None admin] PATH_INFO: `/v3/roles/4ed4b4b515ca4823abbe45566fe27161` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:03.258206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c3392-f15c-48e7-82d5-83a081d6032a None admin] RBAC: Authorizing `identity:get_role(role_id=4ed4b4b515ca4823abbe45566fe27161)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:03.259590 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c3392-f15c-48e7-82d5-83a081d6032a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:03.260937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 561/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:03 2022] GET /identity/v3/roles/4ed4b4b515ca4823abbe45566fe27161 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:03.266884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ada8b-84d2-42f1-b32a-21b66d0594c6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:03.326726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:03.367938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ada8b-84d2-42f1-b32a-21b66d0594c6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:03.404382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994cb92-0bf5-4377-8157-2864d1e740f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:03.405015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994cb92-0bf5-4377-8157-2864d1e740f2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:03.405295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994cb92-0bf5-4377-8157-2864d1e740f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:03.405568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994cb92-0bf5-4377-8157-2864d1e740f2 None admin] PATH_INFO: `/v3/users/4e45f04c076847eeacd492b48a9be268` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:03.419156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994cb92-0bf5-4377-8157-2864d1e740f2 None admin] RBAC: Authorizing `identity:get_user(user_id=4e45f04c076847eeacd492b48a9be268)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:03.420571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994cb92-0bf5-4377-8157-2864d1e740f2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:03.422299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 553/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:03 2022] GET /identity/v3/users/4e45f04c076847eeacd492b48a9be268 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:03.430122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c3392-f15c-48e7-82d5-83a081d6032a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:03.529288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c3392-f15c-48e7-82d5-83a081d6032a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:03.531737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:03.569419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf156-5a10-457a-a5bf-68e846104b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:03.570525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf156-5a10-457a-a5bf-68e846104b20 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:03.570885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf156-5a10-457a-a5bf-68e846104b20 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:03.571220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf156-5a10-457a-a5bf-68e846104b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:03.642398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbf156-5a10-457a-a5bf-68e846104b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:03.643863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbf156-5a10-457a-a5bf-68e846104b20 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:03.753739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 562/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:35:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:35:03.760641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994cb92-0bf5-4377-8157-2864d1e740f2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:03.864901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994cb92-0bf5-4377-8157-2864d1e740f2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:03.881884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:03.899997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc70ee-70e4-416c-8065-123c8bbeb4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:03.900556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc70ee-70e4-416c-8065-123c8bbeb4f2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:03.900836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc70ee-70e4-416c-8065-123c8bbeb4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:03.901111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc70ee-70e4-416c-8065-123c8bbeb4f2 None admin] PATH_INFO: `/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:03.902255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc70ee-70e4-416c-8065-123c8bbeb4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=480a1b8eb4974b33b6a52e5d5c903e08)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:03.903684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc70ee-70e4-416c-8065-123c8bbeb4f2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:03.905356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 554/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:03 2022] GET /identity/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:03.913628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf156-5a10-457a-a5bf-68e846104b20 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:03.917080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:04.014322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf156-5a10-457a-a5bf-68e846104b20 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:04.048146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c92962-edd9-41b6-b9fd-a0864118b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:04.048746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92962-edd9-41b6-b9fd-a0864118b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:04.049025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92962-edd9-41b6-b9fd-a0864118b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:04.049292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92962-edd9-41b6-b9fd-a0864118b9f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:04.050015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c92962-edd9-41b6-b9fd-a0864118b9f4 None admin] RBAC: Adding query filter params (role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=480a1b8eb4974b33b6a52e5d5c903e08, user.id=4e45f04c076847eeacd492b48a9be268) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:04.050351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c92962-edd9-41b6-b9fd-a0864118b9f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:04.051776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c92962-edd9-41b6-b9fd-a0864118b9f4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:04.057799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 563/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:35:03 2022] GET /identity/v3/role_assignments?user.id=4e45f04c076847eeacd492b48a9be268&scope.project.id=480a1b8eb4974b33b6a52e5d5c903e08&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:35:06.266051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629ab25-1375-4cf5-8feb-8a1154b122e6 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:06.266658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629ab25-1375-4cf5-8feb-8a1154b122e6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:06.266658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629ab25-1375-4cf5-8feb-8a1154b122e6 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:06.267971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 555/1118] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:06.273455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2bfe7-29e3-4d20-a493-c3272f12e844 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:06.273889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2bfe7-29e3-4d20-a493-c3272f12e844 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:06.274002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2bfe7-29e3-4d20-a493-c3272f12e844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:06.292416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c4b2bfe7-29e3-4d20-a493-c3272f12e844 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:06.293604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c4b2bfe7-29e3-4d20-a493-c3272f12e844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:06.355656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:06.361606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 564/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:06.636848 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629ab25-1375-4cf5-8feb-8a1154b122e6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:06.739794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629ab25-1375-4cf5-8feb-8a1154b122e6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:06.774198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921b970-f3b4-4655-ab62-1557b80ab006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:06.774837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b970-f3b4-4655-ab62-1557b80ab006 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:06.775119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b970-f3b4-4655-ab62-1557b80ab006 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:06.775394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921b970-f3b4-4655-ab62-1557b80ab006 None admin] PATH_INFO: `/v3/roles/4ed4b4b515ca4823abbe45566fe27161` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:06.776794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921b970-f3b4-4655-ab62-1557b80ab006 None admin] RBAC: Authorizing `identity:get_role(role_id=4ed4b4b515ca4823abbe45566fe27161)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:06.778382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921b970-f3b4-4655-ab62-1557b80ab006 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:06.779726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 556/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:06 2022] GET /identity/v3/roles/4ed4b4b515ca4823abbe45566fe27161 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:06.785616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2bfe7-29e3-4d20-a493-c3272f12e844 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:06.888819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2bfe7-29e3-4d20-a493-c3272f12e844 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:06.922975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c2a82-ed37-4ab2-b146-6330a36026b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:06.923608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c2a82-ed37-4ab2-b146-6330a36026b6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:06.923890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c2a82-ed37-4ab2-b146-6330a36026b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:06.924170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c2a82-ed37-4ab2-b146-6330a36026b6 None admin] PATH_INFO: `/v3/users/4e45f04c076847eeacd492b48a9be268` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:06.925308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c2a82-ed37-4ab2-b146-6330a36026b6 None admin] RBAC: Authorizing `identity:get_user(user_id=4e45f04c076847eeacd492b48a9be268)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:06.926786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c2a82-ed37-4ab2-b146-6330a36026b6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:06.929227 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 565/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:06 2022] GET /identity/v3/users/4e45f04c076847eeacd492b48a9be268 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:06.935408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921b970-f3b4-4655-ab62-1557b80ab006 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:06.989342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:07.035921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921b970-f3b4-4655-ab62-1557b80ab006 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:07.070427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8aafe-fbef-4c04-bc33-de49220bf9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:07.071055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8aafe-fbef-4c04-bc33-de49220bf9e0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:07.071337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8aafe-fbef-4c04-bc33-de49220bf9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:07.071611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8aafe-fbef-4c04-bc33-de49220bf9e0 None admin] PATH_INFO: `/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:07.072736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8aafe-fbef-4c04-bc33-de49220bf9e0 None admin] RBAC: Authorizing `identity:get_project(project_id=480a1b8eb4974b33b6a52e5d5c903e08)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:07.074182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8aafe-fbef-4c04-bc33-de49220bf9e0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:07.075842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 557/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:06 2022] GET /identity/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:07.081983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c2a82-ed37-4ab2-b146-6330a36026b6 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:07.147400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:07.187757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c2a82-ed37-4ab2-b146-6330a36026b6 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:07.222655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26483f-160c-42ae-a99e-9cce7fa555a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:07.223314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26483f-160c-42ae-a99e-9cce7fa555a1 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:07.223598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26483f-160c-42ae-a99e-9cce7fa555a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:07.223875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26483f-160c-42ae-a99e-9cce7fa555a1 None admin] PATH_INFO: `/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08/users/4e45f04c076847eeacd492b48a9be268/roles/4ed4b4b515ca4823abbe45566fe27161` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:07.226640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26483f-160c-42ae-a99e-9cce7fa555a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=480a1b8eb4974b33b6a52e5d5c903e08, user_id=4e45f04c076847eeacd492b48a9be268, role_id=4ed4b4b515ca4823abbe45566fe27161)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:07.228090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26483f-160c-42ae-a99e-9cce7fa555a1 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:07.333131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 566/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:35:07 2022] PUT /identity/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08/users/4e45f04c076847eeacd492b48a9be268/roles/4ed4b4b515ca4823abbe45566fe27161 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:35:09.509439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fea34-7298-4133-987d-574591e80909 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:09.510000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fea34-7298-4133-987d-574591e80909 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:09.510000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fea34-7298-4133-987d-574591e80909 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:09.511353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 558/1124] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:09.516896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4fc82-62b8-483f-b58d-49f9ff45c015 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:09.517355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4fc82-62b8-483f-b58d-49f9ff45c015 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:09.517450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4fc82-62b8-483f-b58d-49f9ff45c015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:09.535730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e7c4fc82-62b8-483f-b58d-49f9ff45c015 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:09.536940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e7c4fc82-62b8-483f-b58d-49f9ff45c015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:09.606147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 567/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:09.882059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fea34-7298-4133-987d-574591e80909 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:09.983095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fea34-7298-4133-987d-574591e80909 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:10.003268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:10.016683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa34d-93d1-45b3-98a7-301d56b18f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:10.017322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa34d-93d1-45b3-98a7-301d56b18f91 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:10.017605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa34d-93d1-45b3-98a7-301d56b18f91 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:10.017880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa34d-93d1-45b3-98a7-301d56b18f91 None admin] PATH_INFO: `/v3/roles/4ed4b4b515ca4823abbe45566fe27161` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:10.019421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa34d-93d1-45b3-98a7-301d56b18f91 None admin] RBAC: Authorizing `identity:get_role(role_id=4ed4b4b515ca4823abbe45566fe27161)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:10.020817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa34d-93d1-45b3-98a7-301d56b18f91 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:10.022349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 559/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:09 2022] GET /identity/v3/roles/4ed4b4b515ca4823abbe45566fe27161 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:10.029793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4fc82-62b8-483f-b58d-49f9ff45c015 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:10.086708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:10.136635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4fc82-62b8-483f-b58d-49f9ff45c015 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:10.171395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32778a2-8587-4d6a-ae93-7756f56585c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:10.172023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32778a2-8587-4d6a-ae93-7756f56585c3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:10.172302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32778a2-8587-4d6a-ae93-7756f56585c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:10.172574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32778a2-8587-4d6a-ae93-7756f56585c3 None admin] PATH_INFO: `/v3/users/4e45f04c076847eeacd492b48a9be268` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:10.173707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32778a2-8587-4d6a-ae93-7756f56585c3 None admin] RBAC: Authorizing `identity:get_user(user_id=4e45f04c076847eeacd492b48a9be268)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:10.175166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32778a2-8587-4d6a-ae93-7756f56585c3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:10.176762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 568/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:10 2022] GET /identity/v3/users/4e45f04c076847eeacd492b48a9be268 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:10.183116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa34d-93d1-45b3-98a7-301d56b18f91 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:10.281473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa34d-93d1-45b3-98a7-301d56b18f91 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:10.314844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a76a-37c9-4224-bb7f-0c062ea254da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:10.315442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a76a-37c9-4224-bb7f-0c062ea254da None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:10.315719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a76a-37c9-4224-bb7f-0c062ea254da None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:10.315989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a76a-37c9-4224-bb7f-0c062ea254da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:10.372061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:10.383967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42a76a-37c9-4224-bb7f-0c062ea254da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:10.385419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42a76a-37c9-4224-bb7f-0c062ea254da None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:10.490020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 560/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:35:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:35:10.496075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32778a2-8587-4d6a-ae93-7756f56585c3 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:10.603497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32778a2-8587-4d6a-ae93-7756f56585c3 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:10.627566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:10.649441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998c865-a3a9-49a6-9056-4b8ece52a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:10.649441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998c865-a3a9-49a6-9056-4b8ece52a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:10.649441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998c865-a3a9-49a6-9056-4b8ece52a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:10.649441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998c865-a3a9-49a6-9056-4b8ece52a3c8 None admin] PATH_INFO: `/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:10.649848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998c865-a3a9-49a6-9056-4b8ece52a3c8 None admin] RBAC: Authorizing `identity:get_project(project_id=480a1b8eb4974b33b6a52e5d5c903e08)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:10.649848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998c865-a3a9-49a6-9056-4b8ece52a3c8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:10.649848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 569/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:10 2022] GET /identity/v3/projects/480a1b8eb4974b33b6a52e5d5c903e08 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:10.650329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a76a-37c9-4224-bb7f-0c062ea254da None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:10.739360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:10.753229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a76a-37c9-4224-bb7f-0c062ea254da None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:10.784692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85edbf-b682-44dd-91ca-f599310f6c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:10.785306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85edbf-b682-44dd-91ca-f599310f6c09 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:10.785588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85edbf-b682-44dd-91ca-f599310f6c09 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:10.785862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85edbf-b682-44dd-91ca-f599310f6c09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:10.787137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85edbf-b682-44dd-91ca-f599310f6c09 None admin] RBAC: Adding query filter params (role.id=4ed4b4b515ca4823abbe45566fe27161, scope.project.id=480a1b8eb4974b33b6a52e5d5c903e08, user.id=4e45f04c076847eeacd492b48a9be268) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:10.787440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85edbf-b682-44dd-91ca-f599310f6c09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:10.788870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85edbf-b682-44dd-91ca-f599310f6c09 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:10.794851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 561/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:35:10 2022] GET /identity/v3/role_assignments?user.id=4e45f04c076847eeacd492b48a9be268&scope.project.id=480a1b8eb4974b33b6a52e5d5c903e08&role.id=4ed4b4b515ca4823abbe45566fe27161 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:13.352552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa73c-f1af-43dc-97bb-b17ef83b8d40 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:13.353095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa73c-f1af-43dc-97bb-b17ef83b8d40 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:13.353187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa73c-f1af-43dc-97bb-b17ef83b8d40 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:13.354560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 570/1131] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:13.360113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db5786-1b8f-45c5-9555-b7efecece597 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:13.360560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db5786-1b8f-45c5-9555-b7efecece597 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:13.360669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db5786-1b8f-45c5-9555-b7efecece597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:13.380919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-10db5786-1b8f-45c5-9555-b7efecece597 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:13.382191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-10db5786-1b8f-45c5-9555-b7efecece597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:13.453457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 562/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:13.725218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa73c-f1af-43dc-97bb-b17ef83b8d40 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:13.828292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa73c-f1af-43dc-97bb-b17ef83b8d40 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:13.866593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e4e60-06f1-4989-88ef-50fae8f88177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:13.867224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e4e60-06f1-4989-88ef-50fae8f88177 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:13.867502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e4e60-06f1-4989-88ef-50fae8f88177 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:13.867769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e4e60-06f1-4989-88ef-50fae8f88177 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:13.868879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e4e60-06f1-4989-88ef-50fae8f88177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:13.870432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e4e60-06f1-4989-88ef-50fae8f88177 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:13.872084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 571/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:35:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:13.878024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db5786-1b8f-45c5-9555-b7efecece597 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:13.957849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:13.977050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db5786-1b8f-45c5-9555-b7efecece597 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:14.011672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b957-4309-4985-9b41-0845f4bd2730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:14.012285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b957-4309-4985-9b41-0845f4bd2730 None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:14.012562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b957-4309-4985-9b41-0845f4bd2730 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:14.012834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b957-4309-4985-9b41-0845f4bd2730 None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:14.013482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0b957-4309-4985-9b41-0845f4bd2730 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:14.014937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0b957-4309-4985-9b41-0845f4bd2730 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:14.129623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 563/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:35:13 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:35:16.303216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e6928-300e-4aee-8528-50ace4a31c2f None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:16.303762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e6928-300e-4aee-8528-50ace4a31c2f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:16.303762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e6928-300e-4aee-8528-50ace4a31c2f None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:16.305088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 572/1135] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:16.310568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592aeda-08ab-41de-ab35-ae0a79cdc0cd None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:16.310990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592aeda-08ab-41de-ab35-ae0a79cdc0cd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:16.311108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592aeda-08ab-41de-ab35-ae0a79cdc0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:16.329817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6592aeda-08ab-41de-ab35-ae0a79cdc0cd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:16.331042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6592aeda-08ab-41de-ab35-ae0a79cdc0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:16.396545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 564/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:16.669514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e6928-300e-4aee-8528-50ace4a31c2f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:16.744334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:16.792147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e6928-300e-4aee-8528-50ace4a31c2f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:16.827328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf75bc-41fa-4a24-a945-304449d4ce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:16.827959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf75bc-41fa-4a24-a945-304449d4ce91 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:16.828241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf75bc-41fa-4a24-a945-304449d4ce91 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:16.828514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf75bc-41fa-4a24-a945-304449d4ce91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:16.829626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf75bc-41fa-4a24-a945-304449d4ce91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:16.831110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf75bc-41fa-4a24-a945-304449d4ce91 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:16.832738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 573/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:35:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:16.838729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592aeda-08ab-41de-ab35-ae0a79cdc0cd None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:16.901142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:16.938750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592aeda-08ab-41de-ab35-ae0a79cdc0cd None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:16.975211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80e23-87da-4ef9-bf06-e3fd0ed187ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:16.975931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80e23-87da-4ef9-bf06-e3fd0ed187ba None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:16.976211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80e23-87da-4ef9-bf06-e3fd0ed187ba None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:16.976496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80e23-87da-4ef9-bf06-e3fd0ed187ba None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:16.977147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b80e23-87da-4ef9-bf06-e3fd0ed187ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:16.978616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b80e23-87da-4ef9-bf06-e3fd0ed187ba None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:17.075305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 565/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:35:16 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:35:19.300648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90beeb9-5050-4e00-ac8f-1a1ebcf6ccda None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:19.301183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90beeb9-5050-4e00-ac8f-1a1ebcf6ccda None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:19.301183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90beeb9-5050-4e00-ac8f-1a1ebcf6ccda None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:19.303053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 574/1139] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:19.308354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b7af2-942f-4c01-9924-1a21f073dbf3 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:19.308779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b7af2-942f-4c01-9924-1a21f073dbf3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:19.308892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b7af2-942f-4c01-9924-1a21f073dbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:19.327564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e34b7af2-942f-4c01-9924-1a21f073dbf3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:19.328767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e34b7af2-942f-4c01-9924-1a21f073dbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:19.402727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 566/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:19.673086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90beeb9-5050-4e00-ac8f-1a1ebcf6ccda None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:19.772599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90beeb9-5050-4e00-ac8f-1a1ebcf6ccda None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:19.798762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:19.807918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dca1ff-aa00-4e22-9a5e-0e6066f0e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:19.808542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dca1ff-aa00-4e22-9a5e-0e6066f0e147 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:19.808816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dca1ff-aa00-4e22-9a5e-0e6066f0e147 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:19.809085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dca1ff-aa00-4e22-9a5e-0e6066f0e147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:19.819269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dca1ff-aa00-4e22-9a5e-0e6066f0e147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-d8dca1ff-aa00-4e22-9a5e-0e6066f0e147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:35:19.824212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:35:19.825363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:35:19.826487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 575/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:35:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:35:19.831527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b7af2-942f-4c01-9924-1a21f073dbf3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:19.880544 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:19.930527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b7af2-942f-4c01-9924-1a21f073dbf3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:19.963391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:19.964239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:19.964517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:19.964784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:19.965436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:19.965735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:19.967120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:19.975509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 567/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:35:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:19.981584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dca1ff-aa00-4e22-9a5e-0e6066f0e147 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:20.080126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dca1ff-aa00-4e22-9a5e-0e6066f0e147 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:20.114735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24253be3-998b-4acb-a67a-4bd1d734a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:20.115354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24253be3-998b-4acb-a67a-4bd1d734a348 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:20.115634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24253be3-998b-4acb-a67a-4bd1d734a348 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:20.115904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24253be3-998b-4acb-a67a-4bd1d734a348 None admin] PATH_INFO: `/v3/users/e71382747eba4e7dab24159815a163e0` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:20.128572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24253be3-998b-4acb-a67a-4bd1d734a348 None admin] RBAC: Authorizing `identity:get_user(user_id=e71382747eba4e7dab24159815a163e0)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:20.130026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24253be3-998b-4acb-a67a-4bd1d734a348 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:20.131644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 576/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:19 2022] GET /identity/v3/users/e71382747eba4e7dab24159815a163e0 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:20.137454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:20.234584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daf432-e0d7-4ee1-a6d3-7122c1f9ad84 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:20.246701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:20.266882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a546b9-f422-41e1-aa3e-a3942ecd9f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:20.267478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a546b9-f422-41e1-aa3e-a3942ecd9f45 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:20.267754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a546b9-f422-41e1-aa3e-a3942ecd9f45 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:20.268026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a546b9-f422-41e1-aa3e-a3942ecd9f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:20.335406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a546b9-f422-41e1-aa3e-a3942ecd9f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:20.336841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a546b9-f422-41e1-aa3e-a3942ecd9f45 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:20.431648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 568/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:35:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:35:20.437642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24253be3-998b-4acb-a67a-4bd1d734a348 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:20.484674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:20.539136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24253be3-998b-4acb-a67a-4bd1d734a348 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:20.573280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f02e16-b808-4c21-a6c0-aca1d56b778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:20.573880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02e16-b808-4c21-a6c0-aca1d56b778c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:20.574185 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02e16-b808-4c21-a6c0-aca1d56b778c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:20.574463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02e16-b808-4c21-a6c0-aca1d56b778c None admin] PATH_INFO: `/v3/projects/e09578f028aa46ab92b47c21039aed28` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:20.575597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f02e16-b808-4c21-a6c0-aca1d56b778c None admin] RBAC: Authorizing `identity:get_project(project_id=e09578f028aa46ab92b47c21039aed28)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:20.577038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f02e16-b808-4c21-a6c0-aca1d56b778c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:20.578712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 577/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:20 2022] GET /identity/v3/projects/e09578f028aa46ab92b47c21039aed28 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:20.585579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a546b9-f422-41e1-aa3e-a3942ecd9f45 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:20.601416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:20.708816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a546b9-f422-41e1-aa3e-a3942ecd9f45 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:20.746484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f306440-25f7-4b03-b203-a655a9470a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:20.747003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306440-25f7-4b03-b203-a655a9470a70 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:20.747286 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306440-25f7-4b03-b203-a655a9470a70 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:20.747559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306440-25f7-4b03-b203-a655a9470a70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:20.748285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f306440-25f7-4b03-b203-a655a9470a70 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=e09578f028aa46ab92b47c21039aed28, user.id=e71382747eba4e7dab24159815a163e0) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:20.748597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f306440-25f7-4b03-b203-a655a9470a70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:20.750037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f306440-25f7-4b03-b203-a655a9470a70 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:20.756700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 569/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:35:20 2022] GET /identity/v3/role_assignments?user.id=e71382747eba4e7dab24159815a163e0&scope.project.id=e09578f028aa46ab92b47c21039aed28&role.id=97f11873550e41dba0cd095afb1dec00 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:22.965393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1051da-bc0f-418d-a399-91737074b652 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:22.965940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1051da-bc0f-418d-a399-91737074b652 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:22.965940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1051da-bc0f-418d-a399-91737074b652 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:22.967221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 578/1147] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:22.972535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c8d6b-4111-4be9-8d73-b0e654550a00 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:22.972958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c8d6b-4111-4be9-8d73-b0e654550a00 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:22.973084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c8d6b-4111-4be9-8d73-b0e654550a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:22.991415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-fa0c8d6b-4111-4be9-8d73-b0e654550a00 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:22.992600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fa0c8d6b-4111-4be9-8d73-b0e654550a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:23.063930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 570/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:23.338716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1051da-bc0f-418d-a399-91737074b652 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:23.439584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1051da-bc0f-418d-a399-91737074b652 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:23.452947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:23.472772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282ddd2-7abe-413a-8509-d2f31a5608ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:23.473912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282ddd2-7abe-413a-8509-d2f31a5608ba None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:23.474237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282ddd2-7abe-413a-8509-d2f31a5608ba None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:23.474668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282ddd2-7abe-413a-8509-d2f31a5608ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:23.484468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282ddd2-7abe-413a-8509-d2f31a5608ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-9282ddd2-7abe-413a-8509-d2f31a5608ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:35:23.489905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:35:23.491159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:35:23.492341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 579/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:35:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:35:23.497384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c8d6b-4111-4be9-8d73-b0e654550a00 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:23.505637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:23.595309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c8d6b-4111-4be9-8d73-b0e654550a00 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:23.628991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:23.629601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:23.629903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:23.630211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:23.630881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:23.631180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:23.632541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:23.641076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 571/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:35:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:23.647015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282ddd2-7abe-413a-8509-d2f31a5608ba None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:23.751634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282ddd2-7abe-413a-8509-d2f31a5608ba None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:23.791789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47ce3e-22fa-4e83-8375-5d2281153b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:23.792424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47ce3e-22fa-4e83-8375-5d2281153b95 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:23.792705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47ce3e-22fa-4e83-8375-5d2281153b95 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:23.792978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47ce3e-22fa-4e83-8375-5d2281153b95 None admin] PATH_INFO: `/v3/users/e71382747eba4e7dab24159815a163e0` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:23.794150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47ce3e-22fa-4e83-8375-5d2281153b95 None admin] RBAC: Authorizing `identity:get_user(user_id=e71382747eba4e7dab24159815a163e0)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:23.795677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47ce3e-22fa-4e83-8375-5d2281153b95 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:23.798177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 580/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:23 2022] GET /identity/v3/users/e71382747eba4e7dab24159815a163e0 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:23.803343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:23.872852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:23.908329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b8112-7b9c-4cce-9b21-08d46ed13006 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:23.945700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b437c-fd3b-4520-98fb-fcaad210be3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:23.946767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b437c-fd3b-4520-98fb-fcaad210be3c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:23.947061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b437c-fd3b-4520-98fb-fcaad210be3c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:23.947531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b437c-fd3b-4520-98fb-fcaad210be3c None admin] PATH_INFO: `/v3/projects/e09578f028aa46ab92b47c21039aed28` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:23.949461 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b437c-fd3b-4520-98fb-fcaad210be3c None admin] RBAC: Authorizing `identity:get_project(project_id=e09578f028aa46ab92b47c21039aed28)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:23.951276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b437c-fd3b-4520-98fb-fcaad210be3c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:23.953392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 572/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:23 2022] GET /identity/v3/projects/e09578f028aa46ab92b47c21039aed28 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:35:23.959548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47ce3e-22fa-4e83-8375-5d2281153b95 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:24.005537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:24.062711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47ce3e-22fa-4e83-8375-5d2281153b95 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:24.095746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d318a-93ec-4a53-b4ac-556ae7b748fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:24.096397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d318a-93ec-4a53-b4ac-556ae7b748fe None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:24.096678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d318a-93ec-4a53-b4ac-556ae7b748fe None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:24.096953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d318a-93ec-4a53-b4ac-556ae7b748fe None admin] PATH_INFO: `/v3/projects/e09578f028aa46ab92b47c21039aed28/users/e71382747eba4e7dab24159815a163e0/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:24.099364 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d318a-93ec-4a53-b4ac-556ae7b748fe None admin] RBAC: Authorizing `identity:create_grant(project_id=e09578f028aa46ab92b47c21039aed28, user_id=e71382747eba4e7dab24159815a163e0, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:24.100800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d318a-93ec-4a53-b4ac-556ae7b748fe None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:24.190787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 581/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:35:23 2022] PUT /identity/v3/projects/e09578f028aa46ab92b47c21039aed28/users/e71382747eba4e7dab24159815a163e0/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:35:26.350571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d35d7-f051-414a-9dfb-29a896cf593e None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:26.351186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d35d7-f051-414a-9dfb-29a896cf593e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:26.351186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d35d7-f051-414a-9dfb-29a896cf593e None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:26.352529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 573/1154] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:26.358004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf564b0c-6600-4eda-ab75-7a508e736754 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:26.358484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf564b0c-6600-4eda-ab75-7a508e736754 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:26.358591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf564b0c-6600-4eda-ab75-7a508e736754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:26.377644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-bf564b0c-6600-4eda-ab75-7a508e736754 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:26.378970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bf564b0c-6600-4eda-ab75-7a508e736754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:26.447283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 582/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:26.712840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d35d7-f051-414a-9dfb-29a896cf593e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:26.816832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d35d7-f051-414a-9dfb-29a896cf593e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:26.865150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4c5f2-a5c0-4dcd-b687-473232db58b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:26.866274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c5f2-a5c0-4dcd-b687-473232db58b2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:26.866728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c5f2-a5c0-4dcd-b687-473232db58b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:26.867220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c5f2-a5c0-4dcd-b687-473232db58b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:26.870374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:26.880407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4c5f2-a5c0-4dcd-b687-473232db58b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b7a4c5f2-a5c0-4dcd-b687-473232db58b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:35:26.893197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:35:26.894449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:35:26.895984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 574/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:35:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:35:26.896691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf564b0c-6600-4eda-ab75-7a508e736754 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:26.927579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:26.999073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf564b0c-6600-4eda-ab75-7a508e736754 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:27.041197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:27.041806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:27.042080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:27.042378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:27.043040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:27.043341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:27.044716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:27.053016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 583/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:35:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:27.059166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4c5f2-a5c0-4dcd-b687-473232db58b2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:27.167196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4c5f2-a5c0-4dcd-b687-473232db58b2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:27.209113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800c011-73b8-42cd-be25-d99ea67e46fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:27.209799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c011-73b8-42cd-be25-d99ea67e46fa None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:27.210082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c011-73b8-42cd-be25-d99ea67e46fa None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:27.210394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c011-73b8-42cd-be25-d99ea67e46fa None admin] PATH_INFO: `/v3/users/e71382747eba4e7dab24159815a163e0` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:27.211704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800c011-73b8-42cd-be25-d99ea67e46fa None admin] RBAC: Authorizing `identity:get_user(user_id=e71382747eba4e7dab24159815a163e0)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:27.213195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800c011-73b8-42cd-be25-d99ea67e46fa None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:27.215212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 575/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:27 2022] GET /identity/v3/users/e71382747eba4e7dab24159815a163e0 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:27.221420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:27.326411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:27.333459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdae75-71f9-457a-ab49-38083a60318b None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:27.369838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd71fb-9a28-452b-84f9-4c91a583a90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:27.370559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd71fb-9a28-452b-84f9-4c91a583a90a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:27.370752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd71fb-9a28-452b-84f9-4c91a583a90a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:27.371026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd71fb-9a28-452b-84f9-4c91a583a90a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:27.439614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd71fb-9a28-452b-84f9-4c91a583a90a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:27.440985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd71fb-9a28-452b-84f9-4c91a583a90a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:27.532572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:27.540554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 584/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:35:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:35:27.550734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800c011-73b8-42cd-be25-d99ea67e46fa None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:27.617649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:27.677473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800c011-73b8-42cd-be25-d99ea67e46fa None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:27.713284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9a0c5-2540-4bfc-833b-35f6037085d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:27.714099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a0c5-2540-4bfc-833b-35f6037085d6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:27.714417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a0c5-2540-4bfc-833b-35f6037085d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:27.714701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a0c5-2540-4bfc-833b-35f6037085d6 None admin] PATH_INFO: `/v3/projects/e09578f028aa46ab92b47c21039aed28` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:27.715873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9a0c5-2540-4bfc-833b-35f6037085d6 None admin] RBAC: Authorizing `identity:get_project(project_id=e09578f028aa46ab92b47c21039aed28)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:27.717341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9a0c5-2540-4bfc-833b-35f6037085d6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:27.719123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 576/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:27 2022] GET /identity/v3/projects/e09578f028aa46ab92b47c21039aed28 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:27.726398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd71fb-9a28-452b-84f9-4c91a583a90a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:27.842941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd71fb-9a28-452b-84f9-4c91a583a90a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:27.881810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3916f1d-a577-44f6-8ac1-18ff451ac7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:27.882471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916f1d-a577-44f6-8ac1-18ff451ac7c3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:27.882752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916f1d-a577-44f6-8ac1-18ff451ac7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:27.883025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916f1d-a577-44f6-8ac1-18ff451ac7c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:27.883773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3916f1d-a577-44f6-8ac1-18ff451ac7c3 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=e09578f028aa46ab92b47c21039aed28, user.id=e71382747eba4e7dab24159815a163e0) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:27.884087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3916f1d-a577-44f6-8ac1-18ff451ac7c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:27.885544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3916f1d-a577-44f6-8ac1-18ff451ac7c3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:27.892184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 585/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:35:27 2022] GET /identity/v3/role_assignments?user.id=e71382747eba4e7dab24159815a163e0&scope.project.id=e09578f028aa46ab92b47c21039aed28&role.id=97f11873550e41dba0cd095afb1dec00 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:30.159623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64defb5-c5e7-496c-85c3-bebad4f66353 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:30.160217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64defb5-c5e7-496c-85c3-bebad4f66353 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:30.160217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64defb5-c5e7-496c-85c3-bebad4f66353 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:30.161643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 577/1162] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:30.167427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ebe3-667d-4c81-a5e7-679a1f6c0f14 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:30.167903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ebe3-667d-4c81-a5e7-679a1f6c0f14 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:30.168006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ebe3-667d-4c81-a5e7-679a1f6c0f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:30.186690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-62f4ebe3-667d-4c81-a5e7-679a1f6c0f14 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:30.187898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-62f4ebe3-667d-4c81-a5e7-679a1f6c0f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:30.249990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:30.264898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 586/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:30.532644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64defb5-c5e7-496c-85c3-bebad4f66353 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:30.652573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64defb5-c5e7-496c-85c3-bebad4f66353 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:30.684327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:30.689252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b78e5e-6dc9-4152-936c-c3db8c144995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:30.696306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b78e5e-6dc9-4152-936c-c3db8c144995 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:30.696306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b78e5e-6dc9-4152-936c-c3db8c144995 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:30.696306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b78e5e-6dc9-4152-936c-c3db8c144995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:30.772502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b78e5e-6dc9-4152-936c-c3db8c144995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:30.773984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b78e5e-6dc9-4152-936c-c3db8c144995 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:30.886634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 578/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:35:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:35:30.892908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4ebe3-667d-4c81-a5e7-679a1f6c0f14 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:31.011604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4ebe3-667d-4c81-a5e7-679a1f6c0f14 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:31.058429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c6f08-45a7-4429-9761-7eaaaca6adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:31.059092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c6f08-45a7-4429-9761-7eaaaca6adb0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:31.059373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c6f08-45a7-4429-9761-7eaaaca6adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:31.059651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c6f08-45a7-4429-9761-7eaaaca6adb0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:31.068478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c6f08-45a7-4429-9761-7eaaaca6adb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:31.069962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c6f08-45a7-4429-9761-7eaaaca6adb0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-0f6c6f08-45a7-4429-9761-7eaaaca6adb0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:35:31.083481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:35:31.084834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:35:31.085715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 587/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:35:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:35:31.094811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b78e5e-6dc9-4152-936c-c3db8c144995 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:31.132616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:31.222398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b78e5e-6dc9-4152-936c-c3db8c144995 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:31.276156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43632d90-221d-487f-b5ee-0a79749c442c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:31.277203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632d90-221d-487f-b5ee-0a79749c442c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:31.277671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632d90-221d-487f-b5ee-0a79749c442c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:31.278181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632d90-221d-487f-b5ee-0a79749c442c None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:31.279376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632d90-221d-487f-b5ee-0a79749c442c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:31.279848 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632d90-221d-487f-b5ee-0a79749c442c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:31.281397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632d90-221d-487f-b5ee-0a79749c442c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:31.291165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 579/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 13:35:31 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:33.516809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297fef9-2806-4e1a-893a-2bd8acda63ed None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:33.517370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297fef9-2806-4e1a-893a-2bd8acda63ed None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:33.517370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297fef9-2806-4e1a-893a-2bd8acda63ed None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:33.518743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 588/1167] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:33.524430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef382c57-5104-48a1-a3ea-fed131bb9937 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:33.524857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef382c57-5104-48a1-a3ea-fed131bb9937 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:33.524983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef382c57-5104-48a1-a3ea-fed131bb9937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:33.544225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ef382c57-5104-48a1-a3ea-fed131bb9937 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:33.545411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ef382c57-5104-48a1-a3ea-fed131bb9937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:33.614750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 580/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:33.881478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297fef9-2806-4e1a-893a-2bd8acda63ed None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:33.932663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:33.983856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297fef9-2806-4e1a-893a-2bd8acda63ed None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:34.017830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bee18-465e-4214-a6f9-dd5da41abd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:34.019369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bee18-465e-4214-a6f9-dd5da41abd15 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:34.019369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bee18-465e-4214-a6f9-dd5da41abd15 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:34.019369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bee18-465e-4214-a6f9-dd5da41abd15 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:34.020275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bee18-465e-4214-a6f9-dd5da41abd15 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:34.021547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bee18-465e-4214-a6f9-dd5da41abd15 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:34.123656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 589/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Thu Sep 8 13:35:33 2022] POST /identity/v3/domains => generated 267 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:35:36.310805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2758e-830b-4d78-b2a7-78b3e52d8e91 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:36.311448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2758e-830b-4d78-b2a7-78b3e52d8e91 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:36.311448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2758e-830b-4d78-b2a7-78b3e52d8e91 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:36.312679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 581/1170] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:36.318666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac22f4f-9294-48ac-b744-44d01c7a7b8a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:36.319347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac22f4f-9294-48ac-b744-44d01c7a7b8a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:36.319591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac22f4f-9294-48ac-b744-44d01c7a7b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:36.339355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-fac22f4f-9294-48ac-b744-44d01c7a7b8a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:36.340545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fac22f4f-9294-48ac-b744-44d01c7a7b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:36.410107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 590/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:36.682054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2758e-830b-4d78-b2a7-78b3e52d8e91 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:36.709492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:36.790711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2758e-830b-4d78-b2a7-78b3e52d8e91 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:36.823897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268a730-0889-4e6f-9bb2-b3cc3df59366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:36.824533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268a730-0889-4e6f-9bb2-b3cc3df59366 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:36.824821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268a730-0889-4e6f-9bb2-b3cc3df59366 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:36.825092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268a730-0889-4e6f-9bb2-b3cc3df59366 None admin] PATH_INFO: `/v3/domains/b48f6b05e3c244c2940411aa655a3533` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:36.836151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268a730-0889-4e6f-9bb2-b3cc3df59366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b48f6b05e3c244c2940411aa655a3533)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:36.837599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268a730-0889-4e6f-9bb2-b3cc3df59366 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:36.839562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 582/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 8 13:35:36 2022] GET /identity/v3/domains/b48f6b05e3c244c2940411aa655a3533 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:36.845673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac22f4f-9294-48ac-b744-44d01c7a7b8a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:36.887694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:36.953044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac22f4f-9294-48ac-b744-44d01c7a7b8a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:36.996804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64dbcf-d74b-4e4e-b34f-091cc8cc67b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:36.997411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dbcf-d74b-4e4e-b34f-091cc8cc67b4 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:36.997687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dbcf-d74b-4e4e-b34f-091cc8cc67b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:36.997955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dbcf-d74b-4e4e-b34f-091cc8cc67b4 None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:36.998640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64dbcf-d74b-4e4e-b34f-091cc8cc67b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:37.000073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64dbcf-d74b-4e4e-b34f-091cc8cc67b4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:37.095980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 591/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:35:36 2022] POST /identity/v3/projects => generated 361 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:35:39.359689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79705-2765-4d05-aa57-0d22463e2c3f None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:39.360272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79705-2765-4d05-aa57-0d22463e2c3f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:39.360272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79705-2765-4d05-aa57-0d22463e2c3f None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:39.361565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 583/1174] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:39.367566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b0015-504c-42c1-9662-fb491e09c650 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:39.367990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b0015-504c-42c1-9662-fb491e09c650 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:39.368115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b0015-504c-42c1-9662-fb491e09c650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:39.386851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e86b0015-504c-42c1-9662-fb491e09c650 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:39.388041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e86b0015-504c-42c1-9662-fb491e09c650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:39.465175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 592/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:39.737572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e79705-2765-4d05-aa57-0d22463e2c3f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:39.822504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:39.837931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e79705-2765-4d05-aa57-0d22463e2c3f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:39.872606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece6bf5-9df8-4d88-bd45-a9d113281f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:39.873251 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece6bf5-9df8-4d88-bd45-a9d113281f2c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:39.873531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece6bf5-9df8-4d88-bd45-a9d113281f2c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:39.873803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece6bf5-9df8-4d88-bd45-a9d113281f2c None admin] PATH_INFO: `/v3/domains/b48f6b05e3c244c2940411aa655a3533` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:39.875016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece6bf5-9df8-4d88-bd45-a9d113281f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=b48f6b05e3c244c2940411aa655a3533)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:39.876432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece6bf5-9df8-4d88-bd45-a9d113281f2c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:39.878108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 584/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 8 13:35:39 2022] GET /identity/v3/domains/b48f6b05e3c244c2940411aa655a3533 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:39.884043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b0015-504c-42c1-9662-fb491e09c650 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:39.905756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:39.983446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b0015-504c-42c1-9662-fb491e09c650 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:40.019729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e514c-11b2-4e1e-876d-a0394551ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:40.020319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e514c-11b2-4e1e-876d-a0394551ce5f None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:40.020596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e514c-11b2-4e1e-876d-a0394551ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:40.020864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e514c-11b2-4e1e-876d-a0394551ce5f None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:40.021534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e514c-11b2-4e1e-876d-a0394551ce5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:40.023347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e514c-11b2-4e1e-876d-a0394551ce5f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:40.145461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 593/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:35:39 2022] POST /identity/v3/users => generated 311 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:35:42.384129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df124fa1-19ba-481a-970f-896d6dff4e6f None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:42.384702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df124fa1-19ba-481a-970f-896d6dff4e6f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:42.384702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df124fa1-19ba-481a-970f-896d6dff4e6f None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:42.386000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 585/1178] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:35:42.391297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794fad-9a4d-4ac3-a1b9-b3b3c2f1fadf None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:42.391713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794fad-9a4d-4ac3-a1b9-b3b3c2f1fadf None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:42.391851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794fad-9a4d-4ac3-a1b9-b3b3c2f1fadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:42.411189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-07794fad-9a4d-4ac3-a1b9-b3b3c2f1fadf None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:42.412384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-07794fad-9a4d-4ac3-a1b9-b3b3c2f1fadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:42.480305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 594/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:42.751749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df124fa1-19ba-481a-970f-896d6dff4e6f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:42.853969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df124fa1-19ba-481a-970f-896d6dff4e6f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:42.905874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f771720-9d0c-467d-be22-c206078ea7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:42.906864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f771720-9d0c-467d-be22-c206078ea7a0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:42.907351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f771720-9d0c-467d-be22-c206078ea7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:42.907821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f771720-9d0c-467d-be22-c206078ea7a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:42.922197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f771720-9d0c-467d-be22-c206078ea7a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-4f771720-9d0c-467d-be22-c206078ea7a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:35:42.927084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:35:42.928667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:35:42.930291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 586/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:35:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:35:42.934493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07794fad-9a4d-4ac3-a1b9-b3b3c2f1fadf None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:42.943044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:43.048322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07794fad-9a4d-4ac3-a1b9-b3b3c2f1fadf None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:43.086060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:43.086692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:43.086976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:43.087249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:43.087919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:43.088303 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:43.089943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:43.098464 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 595/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:35:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:35:43.105627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f771720-9d0c-467d-be22-c206078ea7a0 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:43.122434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:43.210634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f771720-9d0c-467d-be22-c206078ea7a0 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:43.255946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc96e6-f3b0-4318-be95-06fa7cd07ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:43.256628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc96e6-f3b0-4318-be95-06fa7cd07ca0 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:43.256909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc96e6-f3b0-4318-be95-06fa7cd07ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:43.257183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc96e6-f3b0-4318-be95-06fa7cd07ca0 None admin] PATH_INFO: `/v3/users/9cefa8e165b84f2eb78c0a830e7e3cb2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:43.272769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc96e6-f3b0-4318-be95-06fa7cd07ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=9cefa8e165b84f2eb78c0a830e7e3cb2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:43.274276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc96e6-f3b0-4318-be95-06fa7cd07ca0 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:43.275976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 587/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:43 2022] GET /identity/v3/users/9cefa8e165b84f2eb78c0a830e7e3cb2 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:35:43.281922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:43.350531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:43.387012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be510845-fd80-4fc6-bc2a-46fce0913c66 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:43.423771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a079ae6-58e7-4cd1-a607-c871bf00df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:43.424581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a079ae6-58e7-4cd1-a607-c871bf00df33 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:43.424673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a079ae6-58e7-4cd1-a607-c871bf00df33 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:43.424893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a079ae6-58e7-4cd1-a607-c871bf00df33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:43.494645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a079ae6-58e7-4cd1-a607-c871bf00df33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:43.495993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a079ae6-58e7-4cd1-a607-c871bf00df33 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:43.564639 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:43.606431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 596/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:35:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:35:43.613185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc96e6-f3b0-4318-be95-06fa7cd07ca0 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:43.692227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:43.723306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc96e6-f3b0-4318-be95-06fa7cd07ca0 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:43.761519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3f62f-eb78-455f-a970-d5936fc6cfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:43.762639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f62f-eb78-455f-a970-d5936fc6cfb5 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:43.763096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f62f-eb78-455f-a970-d5936fc6cfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:43.763571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f62f-eb78-455f-a970-d5936fc6cfb5 None admin] PATH_INFO: `/v3/projects/5f8d41a7c6684000aad4a742d824f599` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:43.765241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3f62f-eb78-455f-a970-d5936fc6cfb5 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8d41a7c6684000aad4a742d824f599)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:43.766717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3f62f-eb78-455f-a970-d5936fc6cfb5 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:43.768399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 588/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:43 2022] GET /identity/v3/projects/5f8d41a7c6684000aad4a742d824f599 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:43.778108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a079ae6-58e7-4cd1-a607-c871bf00df33 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:43.884048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a079ae6-58e7-4cd1-a607-c871bf00df33 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:43.918022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0843a8c-6554-4e24-8a69-95cd3ec1821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:43.918645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843a8c-6554-4e24-8a69-95cd3ec1821d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:43.918921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843a8c-6554-4e24-8a69-95cd3ec1821d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:43.919190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843a8c-6554-4e24-8a69-95cd3ec1821d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:43.919911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0843a8c-6554-4e24-8a69-95cd3ec1821d None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=5f8d41a7c6684000aad4a742d824f599, user.id=9cefa8e165b84f2eb78c0a830e7e3cb2) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:43.920209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0843a8c-6554-4e24-8a69-95cd3ec1821d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:43.921621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0843a8c-6554-4e24-8a69-95cd3ec1821d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:43.927655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 597/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:35:43 2022] GET /identity/v3/role_assignments?user.id=9cefa8e165b84f2eb78c0a830e7e3cb2&scope.project.id=5f8d41a7c6684000aad4a742d824f599&role.id=97f11873550e41dba0cd095afb1dec00 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:46.063882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487abd-c573-48be-ac45-7528a49f5134 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:46.064399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487abd-c573-48be-ac45-7528a49f5134 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:46.064399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487abd-c573-48be-ac45-7528a49f5134 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:46.065681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 589/1186] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:35:46.073071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec724-9349-4c8f-bf09-06b17538ee2c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:46.073602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec724-9349-4c8f-bf09-06b17538ee2c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:46.073602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec724-9349-4c8f-bf09-06b17538ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:46.096240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-232ec724-9349-4c8f-bf09-06b17538ee2c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:46.097566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-232ec724-9349-4c8f-bf09-06b17538ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:46.125099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:46.173906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 598/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:46.442201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01487abd-c573-48be-ac45-7528a49f5134 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:46.557467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01487abd-c573-48be-ac45-7528a49f5134 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:46.595700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7387267-1f9c-4189-91a4-0666617fd26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:46.596335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7387267-1f9c-4189-91a4-0666617fd26e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:46.596618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7387267-1f9c-4189-91a4-0666617fd26e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:46.596888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7387267-1f9c-4189-91a4-0666617fd26e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:46.605216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:46.608237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7387267-1f9c-4189-91a4-0666617fd26e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f7387267-1f9c-4189-91a4-0666617fd26e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:35:46.613117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:35:46.614267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:35:46.615162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 590/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:35:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:35:46.621785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec724-9349-4c8f-bf09-06b17538ee2c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:46.726596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec724-9349-4c8f-bf09-06b17538ee2c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:46.763010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:46.763625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:46.763904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:46.764175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:46.764839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:46.765142 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:46.766657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:46.775836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 599/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:35:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:46.782450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7387267-1f9c-4189-91a4-0666617fd26e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:46.893032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7387267-1f9c-4189-91a4-0666617fd26e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:46.929505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f597-3d29-43a9-a0a7-234d1f562a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:46.930124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f597-3d29-43a9-a0a7-234d1f562a2e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:46.930432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f597-3d29-43a9-a0a7-234d1f562a2e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:46.930700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f597-3d29-43a9-a0a7-234d1f562a2e None admin] PATH_INFO: `/v3/users/9cefa8e165b84f2eb78c0a830e7e3cb2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:46.931814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5f597-3d29-43a9-a0a7-234d1f562a2e None admin] RBAC: Authorizing `identity:get_user(user_id=9cefa8e165b84f2eb78c0a830e7e3cb2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:46.933248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5f597-3d29-43a9-a0a7-234d1f562a2e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:46.934842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 591/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:46 2022] GET /identity/v3/users/9cefa8e165b84f2eb78c0a830e7e3cb2 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:46.941003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:46.973365 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:47.056000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4976-e9b8-4978-af3c-498410bb5116 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:47.090217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c5fd2-2f1f-41af-94c7-fcaf4443bcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:47.091250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5fd2-2f1f-41af-94c7-fcaf4443bcc3 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:47.091250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5fd2-2f1f-41af-94c7-fcaf4443bcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:47.091535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5fd2-2f1f-41af-94c7-fcaf4443bcc3 None admin] PATH_INFO: `/v3/projects/5f8d41a7c6684000aad4a742d824f599` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:47.092514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c5fd2-2f1f-41af-94c7-fcaf4443bcc3 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8d41a7c6684000aad4a742d824f599)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:47.093961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c5fd2-2f1f-41af-94c7-fcaf4443bcc3 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:47.095642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 600/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:46 2022] GET /identity/v3/projects/5f8d41a7c6684000aad4a742d824f599 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:35:47.102647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f597-3d29-43a9-a0a7-234d1f562a2e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:47.172832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:47.206863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f597-3d29-43a9-a0a7-234d1f562a2e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:47.245612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a194c4-27ac-4523-aff1-ce06655d30eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:47.246283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a194c4-27ac-4523-aff1-ce06655d30eb None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:47.246564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a194c4-27ac-4523-aff1-ce06655d30eb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:47.246834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a194c4-27ac-4523-aff1-ce06655d30eb None admin] PATH_INFO: `/v3/projects/5f8d41a7c6684000aad4a742d824f599/users/9cefa8e165b84f2eb78c0a830e7e3cb2/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:47.248705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a194c4-27ac-4523-aff1-ce06655d30eb None admin] RBAC: Authorizing `identity:create_grant(project_id=5f8d41a7c6684000aad4a742d824f599, user_id=9cefa8e165b84f2eb78c0a830e7e3cb2, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:47.250129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a194c4-27ac-4523-aff1-ce06655d30eb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:47.348930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 592/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:35:47 2022] PUT /identity/v3/projects/5f8d41a7c6684000aad4a742d824f599/users/9cefa8e165b84f2eb78c0a830e7e3cb2/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:35:49.538364 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66680e8-0962-4e6a-bb3a-d70eb46bf241 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:49.538913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66680e8-0962-4e6a-bb3a-d70eb46bf241 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:49.538913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66680e8-0962-4e6a-bb3a-d70eb46bf241 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:49.540288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 601/1193] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:35:49.545986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b636-63f6-4113-b2e8-3ccf1437c93c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:49.546465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b636-63f6-4113-b2e8-3ccf1437c93c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:49.546585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b636-63f6-4113-b2e8-3ccf1437c93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:49.567092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-04e1b636-63f6-4113-b2e8-3ccf1437c93c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:35:49.568312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-04e1b636-63f6-4113-b2e8-3ccf1437c93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:35:49.643010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 593/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:35:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:35:49.932384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66680e8-0962-4e6a-bb3a-d70eb46bf241 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:50.038759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:50.048668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66680e8-0962-4e6a-bb3a-d70eb46bf241 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:50.084009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278fbe6-a3db-48da-b7ad-1712cff252b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:50.084655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fbe6-a3db-48da-b7ad-1712cff252b8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:50.085219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fbe6-a3db-48da-b7ad-1712cff252b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:50.085858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fbe6-a3db-48da-b7ad-1712cff252b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:50.096672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278fbe6-a3db-48da-b7ad-1712cff252b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c278fbe6-a3db-48da-b7ad-1712cff252b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:35:50.102179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:35:50.103872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:35:50.105549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 602/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:35:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:35:50.109703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1b636-63f6-4113-b2e8-3ccf1437c93c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:50.178014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:50.236232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1b636-63f6-4113-b2e8-3ccf1437c93c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:50.275962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:50.276572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:50.276854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:50.277132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:50.277838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:50.278180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:50.279552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:50.288746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 594/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:35:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:35:50.295478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278fbe6-a3db-48da-b7ad-1712cff252b8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:50.400928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278fbe6-a3db-48da-b7ad-1712cff252b8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:50.436318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85c34a-bb9b-4690-bb1a-d6c22a39a629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:50.436944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85c34a-bb9b-4690-bb1a-d6c22a39a629 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:50.437221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85c34a-bb9b-4690-bb1a-d6c22a39a629 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:50.437496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85c34a-bb9b-4690-bb1a-d6c22a39a629 None admin] PATH_INFO: `/v3/users/9cefa8e165b84f2eb78c0a830e7e3cb2` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:50.438671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85c34a-bb9b-4690-bb1a-d6c22a39a629 None admin] RBAC: Authorizing `identity:get_user(user_id=9cefa8e165b84f2eb78c0a830e7e3cb2)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:50.440101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85c34a-bb9b-4690-bb1a-d6c22a39a629 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:50.442231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 603/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 13:35:50 2022] GET /identity/v3/users/9cefa8e165b84f2eb78c0a830e7e3cb2 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:50.449183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:50.558446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e2eaa-eac3-436b-9ab4-beb109c143c1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:50.580429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:50.596537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b335c9-c28f-4c8c-b67a-a20854f88526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:50.597188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b335c9-c28f-4c8c-b67a-a20854f88526 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:50.597466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b335c9-c28f-4c8c-b67a-a20854f88526 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:50.597737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b335c9-c28f-4c8c-b67a-a20854f88526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:50.671549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b335c9-c28f-4c8c-b67a-a20854f88526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:50.672999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b335c9-c28f-4c8c-b67a-a20854f88526 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:50.787861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 595/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:35:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:35:50.794048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85c34a-bb9b-4690-bb1a-d6c22a39a629 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:50.818758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:50.908127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85c34a-bb9b-4690-bb1a-d6c22a39a629 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:50.949171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db0ab-a6d6-4a8f-a39c-c7239aaa0d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:50.949806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db0ab-a6d6-4a8f-a39c-c7239aaa0d83 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:50.950085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db0ab-a6d6-4a8f-a39c-c7239aaa0d83 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:50.950383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db0ab-a6d6-4a8f-a39c-c7239aaa0d83 None admin] PATH_INFO: `/v3/projects/5f8d41a7c6684000aad4a742d824f599` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:50.951503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7db0ab-a6d6-4a8f-a39c-c7239aaa0d83 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8d41a7c6684000aad4a742d824f599)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:50.952939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7db0ab-a6d6-4a8f-a39c-c7239aaa0d83 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:50.954633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 604/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 13:35:50 2022] GET /identity/v3/projects/5f8d41a7c6684000aad4a742d824f599 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:35:50.960888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b335c9-c28f-4c8c-b67a-a20854f88526 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:35:50.988520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:35:51.071653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b335c9-c28f-4c8c-b67a-a20854f88526 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:35:51.109443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedd2e3-0641-48ea-a8a4-63f54cfdc85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:35:51.110050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedd2e3-0641-48ea-a8a4-63f54cfdc85c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:35:51.110349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedd2e3-0641-48ea-a8a4-63f54cfdc85c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:35:51.110626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedd2e3-0641-48ea-a8a4-63f54cfdc85c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:35:51.111350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedd2e3-0641-48ea-a8a4-63f54cfdc85c None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=5f8d41a7c6684000aad4a742d824f599, user.id=9cefa8e165b84f2eb78c0a830e7e3cb2) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:35:51.111659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedd2e3-0641-48ea-a8a4-63f54cfdc85c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:35:51.113714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedd2e3-0641-48ea-a8a4-63f54cfdc85c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:35:51.119845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 596/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:35:50 2022] GET /identity/v3/role_assignments?user.id=9cefa8e165b84f2eb78c0a830e7e3cb2&scope.project.id=5f8d41a7c6684000aad4a742d824f599&role.id=97f11873550e41dba0cd095afb1dec00 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:00.944908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=56 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:01.117028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=58 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:11.454493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08317340-94b4-4bf0-963b-5a6f73e5d330 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.455127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08317340-94b4-4bf0-963b-5a6f73e5d330 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.455127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08317340-94b4-4bf0-963b-5a6f73e5d330 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.456713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 605/1201] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:11.462243 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a8ae7-318d-4ca8-9308-3b5a20e41d61 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.462697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a8ae7-318d-4ca8-9308-3b5a20e41d61 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.462795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a8ae7-318d-4ca8-9308-3b5a20e41d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.482687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-1c2a8ae7-318d-4ca8-9308-3b5a20e41d61 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:11.483912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1c2a8ae7-318d-4ca8-9308-3b5a20e41d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:11.556382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 597/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:11.828063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08317340-94b4-4bf0-963b-5a6f73e5d330 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:11.940540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08317340-94b4-4bf0-963b-5a6f73e5d330 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:11.981244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:11.981906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.982215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.982495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.996029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:11.997504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:12.012792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:12.013996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:12.015137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:12.015137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:12.015137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:12.015137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 606/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:12.020844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a8ae7-318d-4ca8-9308-3b5a20e41d61 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.140342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a8ae7-318d-4ca8-9308-3b5a20e41d61 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.170736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:12.176607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53d963-3b5c-4c7a-b827-2202fb3f9c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:12.177230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d963-3b5c-4c7a-b827-2202fb3f9c52 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.177507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d963-3b5c-4c7a-b827-2202fb3f9c52 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.177790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53d963-3b5c-4c7a-b827-2202fb3f9c52 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.178473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53d963-3b5c-4c7a-b827-2202fb3f9c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:12.180052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53d963-3b5c-4c7a-b827-2202fb3f9c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:12.180245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53d963-3b5c-4c7a-b827-2202fb3f9c52 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:12.191875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 598/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:12.198355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.252618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:12.301579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90eec3-c56d-4eb4-ab61-3bde4fcc8240 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.345882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5246432-5e0a-4879-a105-0c49a8bfd480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:12.346519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246432-5e0a-4879-a105-0c49a8bfd480 None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.346801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246432-5e0a-4879-a105-0c49a8bfd480 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.347074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246432-5e0a-4879-a105-0c49a8bfd480 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.347748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246432-5e0a-4879-a105-0c49a8bfd480 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:12.349283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246432-5e0a-4879-a105-0c49a8bfd480 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:12.444476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 607/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:12 2022] POST /identity/v3/users => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:14.681106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77826c-30ec-40db-88ce-e795d853086a None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.681650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77826c-30ec-40db-88ce-e795d853086a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.681650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77826c-30ec-40db-88ce-e795d853086a None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.683025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 599/1206] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:14.688333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e2e1c-d385-4d98-9694-92620d9e5aaf None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.688824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e2e1c-d385-4d98-9694-92620d9e5aaf None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.688961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e2e1c-d385-4d98-9694-92620d9e5aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.707353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a29e2e1c-d385-4d98-9694-92620d9e5aaf None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:14.709073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a29e2e1c-d385-4d98-9694-92620d9e5aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:14.777922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 608/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:15.046117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77826c-30ec-40db-88ce-e795d853086a None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.151599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77826c-30ec-40db-88ce-e795d853086a None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.185930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189878de-2f9b-4198-b36e-d4fde15e2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.186717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189878de-2f9b-4198-b36e-d4fde15e2d51 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.186999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189878de-2f9b-4198-b36e-d4fde15e2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.187272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189878de-2f9b-4198-b36e-d4fde15e2d51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.198639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189878de-2f9b-4198-b36e-d4fde15e2d51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-189878de-2f9b-4198-b36e-d4fde15e2d51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:15.203903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:15.205725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:15.207368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 600/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:15.211502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e2e1c-d385-4d98-9694-92620d9e5aaf None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.254053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.314294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e2e1c-d385-4d98-9694-92620d9e5aaf None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.349859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.350503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.350788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.351063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.351789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:15.352187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.353565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.361950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 609/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:15.368650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189878de-2f9b-4198-b36e-d4fde15e2d51 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.419506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.474620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189878de-2f9b-4198-b36e-d4fde15e2d51 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.509076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.509701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.509983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.510299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.521219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.522745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:15.537405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:15.538676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:15.540132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:15.540132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:15.540132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:15.540132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 601/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:15.544455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.645508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.648461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84a051-417d-4537-a892-efc8c0a0a761 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.683091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.683706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.683986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.684260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.684907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:15.685214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.686743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.700205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 610/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:15.706073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.806970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df23b2-e813-488b-8b89-85ed2fbd565e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.808502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.840326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.841031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.841309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.841609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.849714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.851231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:37:15.862340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:15.863526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:15.865312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 602/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:15.870120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.968231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6af17-9f5a-41e0-93c7-126771602147 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.004732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.005339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.005623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.005900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.006709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:16.007027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.008471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.025625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 611/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:16.031573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.131620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be33d-9f86-44b9-88cb-cd2f8e0b5b47 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.167602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94833ec8-65d1-4815-960f-ad1121f1db5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.168215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94833ec8-65d1-4815-960f-ad1121f1db5e None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.168493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94833ec8-65d1-4815-960f-ad1121f1db5e None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.168833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94833ec8-65d1-4815-960f-ad1121f1db5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.199084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.237329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94833ec8-65d1-4815-960f-ad1121f1db5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.238974 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94833ec8-65d1-4815-960f-ad1121f1db5e None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.338902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 603/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:37:16.345183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.364862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.444497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf6c3b-d0e2-4d6b-ab44-3901d9dd06b5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.478202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.478832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.479114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.479384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.490463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.491856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:16.507124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:16.508348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:16.509278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:16.509278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:16.509278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:16.509278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 612/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:16.517299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94833ec8-65d1-4815-960f-ad1121f1db5e None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.580048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.618618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94833ec8-65d1-4815-960f-ad1121f1db5e None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.652421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.653026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.653302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.653567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.654235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:16.654546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.655871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.666971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 604/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:16.674248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.745654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.777358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be230-ca6f-474e-b7dc-9f21c8c0789c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.817613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.818302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.818863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.818979 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.826851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.828306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:37:16.840591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:16.847587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:16.848747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 613/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:16.855018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.959877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8dbad5-5639-4896-9e70-0f5ca13e85aa None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.983412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.995741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ae329-cbad-4cab-a7e7-6457b34491cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.996279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ae329-cbad-4cab-a7e7-6457b34491cf None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.996563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ae329-cbad-4cab-a7e7-6457b34491cf None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.996840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ae329-cbad-4cab-a7e7-6457b34491cf None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.997541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ae329-cbad-4cab-a7e7-6457b34491cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:16.997852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ae329-cbad-4cab-a7e7-6457b34491cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.999323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ae329-cbad-4cab-a7e7-6457b34491cf None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.012117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 605/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:17.019800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.122783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545594cd-0fc7-4549-beeb-f859ee3bc2f8 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.155636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.161593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebff38-f66b-40b6-a675-e4162a6c465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.162218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebff38-f66b-40b6-a675-e4162a6c465a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.162499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebff38-f66b-40b6-a675-e4162a6c465a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.162771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebff38-f66b-40b6-a675-e4162a6c465a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.163517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebff38-f66b-40b6-a675-e4162a6c465a None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=645282ae0ac7450dbd27d71c467b5216) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:17.163799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebff38-f66b-40b6-a675-e4162a6c465a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.165228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebff38-f66b-40b6-a675-e4162a6c465a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.170530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 614/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/role_assignments?user.id=645282ae0ac7450dbd27d71c467b5216&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:19.302106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cdc57-268f-429d-98e3-aed8fbc41488 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.302708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cdc57-268f-429d-98e3-aed8fbc41488 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.302812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cdc57-268f-429d-98e3-aed8fbc41488 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.304205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 606/1220] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:19.309610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921627f-b0d1-40b6-a33d-66d07670a842 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.310032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921627f-b0d1-40b6-a33d-66d07670a842 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.310206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921627f-b0d1-40b6-a33d-66d07670a842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.330233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-4921627f-b0d1-40b6-a33d-66d07670a842 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:19.331425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4921627f-b0d1-40b6-a33d-66d07670a842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:19.402250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 615/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:37:19.674894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cdc57-268f-429d-98e3-aed8fbc41488 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.776282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cdc57-268f-429d-98e3-aed8fbc41488 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.809840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c314c-4a1b-44a8-97dd-abaa6376d38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.810501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c314c-4a1b-44a8-97dd-abaa6376d38b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.810773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c314c-4a1b-44a8-97dd-abaa6376d38b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.816046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c314c-4a1b-44a8-97dd-abaa6376d38b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.821516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c314c-4a1b-44a8-97dd-abaa6376d38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-425c314c-4a1b-44a8-97dd-abaa6376d38b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:19.826271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:19.828112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:19.829733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 607/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:19.834011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921627f-b0d1-40b6-a33d-66d07670a842 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.931809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921627f-b0d1-40b6-a33d-66d07670a842 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.955817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.965930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.966573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.967121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.967398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.968065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] RBAC: Adding query filter params (name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:19.968368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:19.969807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:19.977918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 616/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:19.983825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c314c-4a1b-44a8-97dd-abaa6376d38b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.028621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.083629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c314c-4a1b-44a8-97dd-abaa6376d38b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.118493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.119117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.119393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.119663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.130441 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.131888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:20.146559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:20.147916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:20.149212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:20.149212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:20.149212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:20.149212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 608/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:20.154664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.259150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be7558-9c0c-472a-9900-ef6318b69a59 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.293728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.294411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.294759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.295035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.295678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:20.295984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.297384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.309833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 617/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.315939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.417620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.424313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d5c69-21a6-4aef-b2da-3d0456811a78 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.458331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.458970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.459250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.459521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.467913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.469348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:37:20.481942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:20.483217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:20.484230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 609/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:20.490413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.539692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.600111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71980d00-028f-4745-8b32-180292b4b3f7 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.635348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.635959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.636238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.636509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.637262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:20.637562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.639014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.657080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 618/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.663997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.781591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c681f-a091-445e-a510-b45f68c5dce6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.822728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.823278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.823561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.823835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.835704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.837160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:20.852474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:20.853767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:20.854920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:20.854920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:20.854920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:20.854920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 610/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:20.859813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.961391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910403b8-9ca0-47d0-a17e-d5e46eb9a4f6 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.963793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.996007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.996610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.996885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.997151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.997789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:20.998087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.999463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.011243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 619/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:21.017088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:21.027073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:21.116248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b7b6a-7cc9-45af-8422-467bcb4821c1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.152476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.153099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.153376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.153646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.160091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.161619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:37:21.173890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:21.175061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:21.176713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 611/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:21.181403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:21.288878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df45172-6920-46ff-8c9e-c82692bb5f4c None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.322588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf780f28-42e7-43a8-a199-cef2fe19815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.323254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780f28-42e7-43a8-a199-cef2fe19815f None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.323458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780f28-42e7-43a8-a199-cef2fe19815f None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.323725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780f28-42e7-43a8-a199-cef2fe19815f None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.324408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780f28-42e7-43a8-a199-cef2fe19815f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:21.324709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780f28-42e7-43a8-a199-cef2fe19815f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.326125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780f28-42e7-43a8-a199-cef2fe19815f None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.338850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 620/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:37:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:21.345104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:21.410109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:21.445598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa7e7c-216d-4a59-9bc4-ff2394fbf98d None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.478805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afc445-ee21-4e86-b68c-0fe66af0c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.479476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afc445-ee21-4e86-b68c-0fe66af0c4ca None admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.479756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afc445-ee21-4e86-b68c-0fe66af0c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.480030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afc445-ee21-4e86-b68c-0fe66af0c4ca None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/645282ae0ac7450dbd27d71c467b5216/roles/04bab2a5f02f4140b88885a57b278b5d` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.495313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afc445-ee21-4e86-b68c-0fe66af0c4ca None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=645282ae0ac7450dbd27d71c467b5216, role_id=04bab2a5f02f4140b88885a57b278b5d)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.497329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afc445-ee21-4e86-b68c-0fe66af0c4ca None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.589057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 612/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:21 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/645282ae0ac7450dbd27d71c467b5216/roles/04bab2a5f02f4140b88885a57b278b5d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:23.775310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ec3c7-0d95-411e-b0c0-58678659cc6d None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.776050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ec3c7-0d95-411e-b0c0-58678659cc6d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.776050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ec3c7-0d95-411e-b0c0-58678659cc6d None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.777211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 621/1233] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:23.783319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6065c-41db-4962-a378-f6c714cad369 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.783811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6065c-41db-4962-a378-f6c714cad369 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.784112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6065c-41db-4962-a378-f6c714cad369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.804226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-74f6065c-41db-4962-a378-f6c714cad369 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:23.807683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-74f6065c-41db-4962-a378-f6c714cad369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:23.872745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 613/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:24.138617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ec3c7-0d95-411e-b0c0-58678659cc6d None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.165736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.244234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ec3c7-0d95-411e-b0c0-58678659cc6d None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.279390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ccc65-d80d-42b6-ab0c-15fc4e93d837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.280019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ccc65-d80d-42b6-ab0c-15fc4e93d837 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.280299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ccc65-d80d-42b6-ab0c-15fc4e93d837 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.280569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ccc65-d80d-42b6-ab0c-15fc4e93d837 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.290473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ccc65-d80d-42b6-ab0c-15fc4e93d837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-027ccc65-d80d-42b6-ab0c-15fc4e93d837 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:24.295361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:24.300216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:24.301326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 622/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:24.302883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6065c-41db-4962-a378-f6c714cad369 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.352621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.408330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6065c-41db-4962-a378-f6c714cad369 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.443325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.443992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.444550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.445052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.446096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:24.446474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.447843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.456132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 614/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:24.462253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ccc65-d80d-42b6-ab0c-15fc4e93d837 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.548288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.564138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ccc65-d80d-42b6-ab0c-15fc4e93d837 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.612051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.612742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.613121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.613449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.625183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.626824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:24.642718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:24.654544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:24.655595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:24.655595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:24.655595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:24.655595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 623/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:24.655847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.751355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd04e1-a5bf-4ff5-af7f-46bfa7cc50e1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.760115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.784210 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.784813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.785087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.785427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.786629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:24.786936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.788294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.800573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 615/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:24.806781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.904273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba433956-2ecb-470f-b4b5-3f41fa05629d None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.938906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.939544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.939856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.940158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.949376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.955181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.956634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:37:24.968139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:24.971342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:24.972589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 624/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:24.975871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.075169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3955d-f1b5-49fc-bceb-1f80b0e20736 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.124134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.125123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.125602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.126073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.127407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:25.127935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.130583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.162017 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 616/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:25.167883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.282298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66d5b9-ea70-42c0-848c-bd4682e74f65 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.316542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277475c-7fc6-4e13-84ac-514378c16cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.317728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277475c-7fc6-4e13-84ac-514378c16cb4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.318113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277475c-7fc6-4e13-84ac-514378c16cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.323083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277475c-7fc6-4e13-84ac-514378c16cb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.380408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.386807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277475c-7fc6-4e13-84ac-514378c16cb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.388287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277475c-7fc6-4e13-84ac-514378c16cb4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.488276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 625/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:37:25.494878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.525530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.596631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae5885-65bb-48f6-8049-782df8a5c3d2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.634524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.635094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.635375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.635686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.647227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.648688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:25.664621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:25.665869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:25.667114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:25.667114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:25.667114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:25.667114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 617/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:25.671682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277475c-7fc6-4e13-84ac-514378c16cb4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.766996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.772694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277475c-7fc6-4e13-84ac-514378c16cb4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.806984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.807651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.807855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.808122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.808763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:25.809062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.810451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.823395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 626/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:25.829223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.909282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.930278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdf81e-3d16-45d1-ac78-420535597819 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.962722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.962722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.962722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.962722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.972569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.976867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:37:25.991029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:25.992108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:25.993062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 618/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:25.999052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.099772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde7752-0c96-46cd-90d5-876d444293e0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.133119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6772d-6ff8-4886-8287-fa86720950cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.133722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6772d-6ff8-4886-8287-fa86720950cf None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.134002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6772d-6ff8-4886-8287-fa86720950cf None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.134306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6772d-6ff8-4886-8287-fa86720950cf None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.134995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6772d-6ff8-4886-8287-fa86720950cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:26.135302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6772d-6ff8-4886-8287-fa86720950cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.136728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6772d-6ff8-4886-8287-fa86720950cf None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.149085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 627/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:26.155184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.256781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e797f-5d3d-4d2d-b9e5-84be91556c59 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.294608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19e1de-1a81-4d72-a634-f9a85a749607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.295213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e1de-1a81-4d72-a634-f9a85a749607 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.295886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e1de-1a81-4d72-a634-f9a85a749607 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.296587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19e1de-1a81-4d72-a634-f9a85a749607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.297294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19e1de-1a81-4d72-a634-f9a85a749607 None admin] RBAC: Adding query filter params (role.id=04bab2a5f02f4140b88885a57b278b5d, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=645282ae0ac7450dbd27d71c467b5216) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:26.297606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19e1de-1a81-4d72-a634-f9a85a749607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.299126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19e1de-1a81-4d72-a634-f9a85a749607 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.306150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 619/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/role_assignments?user.id=645282ae0ac7450dbd27d71c467b5216&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=04bab2a5f02f4140b88885a57b278b5d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:28.534664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3daf29-aa2b-431f-995b-4f8c7c403a5f None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.535234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3daf29-aa2b-431f-995b-4f8c7c403a5f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.535234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3daf29-aa2b-431f-995b-4f8c7c403a5f None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.537107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 628/1247] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:28.542431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75a9d-b6cd-4d2d-b671-b319445d610f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.542912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75a9d-b6cd-4d2d-b671-b319445d610f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.543014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b75a9d-b6cd-4d2d-b671-b319445d610f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.557021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:28.569287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-d2b75a9d-b6cd-4d2d-b671-b319445d610f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:28.570520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d2b75a9d-b6cd-4d2d-b671-b319445d610f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:28.637404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 620/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:28.904673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3daf29-aa2b-431f-995b-4f8c7c403a5f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.923020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.003353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3daf29-aa2b-431f-995b-4f8c7c403a5f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.049305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5418b1-8a83-4804-b9c6-fb00c2df55d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.049932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5418b1-8a83-4804-b9c6-fb00c2df55d5 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.050236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5418b1-8a83-4804-b9c6-fb00c2df55d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.050507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5418b1-8a83-4804-b9c6-fb00c2df55d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.061209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5418b1-8a83-4804-b9c6-fb00c2df55d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-5d5418b1-8a83-4804-b9c6-fb00c2df55d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:29.066127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:29.067135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:29.068007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 629/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:29.073496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b75a9d-b6cd-4d2d-b671-b319445d610f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.176584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b75a9d-b6cd-4d2d-b671-b319445d610f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.205977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.211481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.212090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.212370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.212646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.213310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:29.213615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.215047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.223992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 621/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.230147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5418b1-8a83-4804-b9c6-fb00c2df55d5 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.312062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.335269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5418b1-8a83-4804-b9c6-fb00c2df55d5 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.368238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.368858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.369143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.369416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.380250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.381743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:29.396064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:29.397249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:29.398448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:29.398448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:29.398448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:29.398448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 630/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:29.403099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.505257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96d175-cd38-453e-99f3-07f1441f0fce None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.539639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.540255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.540527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.540797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.541434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:29.541723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.543098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.555053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 622/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.560953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.662122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f243ec6-99fa-48ef-9979-c6db362c6998 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.687628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.695843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.698585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.698585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.698585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] PATH_INFO: `/v3/users/placement` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.704478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.705872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:37:29.717024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:29.718442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:29.719843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 631/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:29.724694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.780193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.824285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3921458-3cfc-4124-873b-ec03f542308c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.857098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.857727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.857993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.858299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.859022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:29.859328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.860755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.878748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 623/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.885190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.985821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b4865-9d39-42b4-9d5b-b5b008990fdf None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.019892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70031854-60c3-4d67-99e8-37155870d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.020536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031854-60c3-4d67-99e8-37155870d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.020810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031854-60c3-4d67-99e8-37155870d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.021079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031854-60c3-4d67-99e8-37155870d7e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.076441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.088290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70031854-60c3-4d67-99e8-37155870d7e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.089746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70031854-60c3-4d67-99e8-37155870d7e7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.186575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 632/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:37:30.192484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.291104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445b5ac-7833-4f67-8043-ccfb5f685728 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.304757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.332192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.332803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.333083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.333358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.343644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.345085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:30.359718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:30.360861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:30.362116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:30.362116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:30.362116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:30.362116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 624/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:30.367264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70031854-60c3-4d67-99e8-37155870d7e7 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.453924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.466006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70031854-60c3-4d67-99e8-37155870d7e7 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.500201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.500810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.501093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.501391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.502029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:30.502367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.503758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.514635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 633/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:30.521004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.620180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9928a9-7ccf-4f32-a96b-ed7f18a71c7a None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.654013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.654671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.654952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.655218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.662841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.664269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:37:30.675235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:30.676471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:30.677551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 625/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:30.682720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.785423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb096e-a55a-48a7-808a-1588e11f5532 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.819229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51c181-9e9b-42f6-a91f-f81e4c8a920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.819836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51c181-9e9b-42f6-a91f-f81e4c8a920a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.820119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51c181-9e9b-42f6-a91f-f81e4c8a920a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.820393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51c181-9e9b-42f6-a91f-f81e4c8a920a None admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.821084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51c181-9e9b-42f6-a91f-f81e4c8a920a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:30.821390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51c181-9e9b-42f6-a91f-f81e4c8a920a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.822851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51c181-9e9b-42f6-a91f-f81e4c8a920a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.835259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 634/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:30.841427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.868217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.940510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a712-0bfd-48af-afc4-6c633e4ada33 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.973812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6590e53-c63b-45cf-a6df-e6c3eb821427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.974434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6590e53-c63b-45cf-a6df-e6c3eb821427 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.974711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6590e53-c63b-45cf-a6df-e6c3eb821427 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.975040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6590e53-c63b-45cf-a6df-e6c3eb821427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.975761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6590e53-c63b-45cf-a6df-e6c3eb821427 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=645282ae0ac7450dbd27d71c467b5216) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:30.976063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6590e53-c63b-45cf-a6df-e6c3eb821427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.977464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6590e53-c63b-45cf-a6df-e6c3eb821427 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.983030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 626/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/role_assignments?user.id=645282ae0ac7450dbd27d71c467b5216&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=97f11873550e41dba0cd095afb1dec00 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:33.122966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d011581-dff0-4478-a7b3-fb963eb51e5e None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.123515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d011581-dff0-4478-a7b3-fb963eb51e5e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.123515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d011581-dff0-4478-a7b3-fb963eb51e5e None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.124829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 635/1261] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:33.130131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc2a6d-2dfb-46b5-a20e-7db9cf14ac12 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.130563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc2a6d-2dfb-46b5-a20e-7db9cf14ac12 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.130735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc2a6d-2dfb-46b5-a20e-7db9cf14ac12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.148624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-35fc2a6d-2dfb-46b5-a20e-7db9cf14ac12 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:33.149866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-35fc2a6d-2dfb-46b5-a20e-7db9cf14ac12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:33.218657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 627/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:33.497769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d011581-dff0-4478-a7b3-fb963eb51e5e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.508790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:33.601340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d011581-dff0-4478-a7b3-fb963eb51e5e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:33.637415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb8501-870e-43e8-8e0d-0ece689e3bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:33.638327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb8501-870e-43e8-8e0d-0ece689e3bd0 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.638632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb8501-870e-43e8-8e0d-0ece689e3bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.638936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb8501-870e-43e8-8e0d-0ece689e3bd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.650880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb8501-870e-43e8-8e0d-0ece689e3bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c6eb8501-870e-43e8-8e0d-0ece689e3bd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:33.655711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:33.656898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:33.657962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 636/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:33.663830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc2a6d-2dfb-46b5-a20e-7db9cf14ac12 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.681982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:33.760334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc2a6d-2dfb-46b5-a20e-7db9cf14ac12 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:33.793906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:33.794545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.794828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.795104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.795771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:33.796078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:33.797439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:33.805390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 628/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:33.811237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb8501-870e-43e8-8e0d-0ece689e3bd0 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.877949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:33.910632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb8501-870e-43e8-8e0d-0ece689e3bd0 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:33.945130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:33.945760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.946041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.946344 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.956409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:33.958574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:33.973817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:33.975026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:33.975955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:33.975955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:33.975955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:33.975955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 637/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:33.980844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.061601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.081229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab126f2f-d1f0-4840-ad6e-88f6cd6eb1d6 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.113822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.115000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.115279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.115551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.116205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:34.116504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.117863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.130286 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 629/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:34.136615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.241109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c0ed-06d0-469f-a988-4451fc2a9c17 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.261071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.274564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.275216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.275477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.275754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.283977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.285425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:37:34.296332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:34.297479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:34.298835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 638/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:34.303819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.401866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e05eb-237a-4276-b243-e2ee1604e6f2 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.436170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.436772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.437058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.437329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.438204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:34.438454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.445589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.448892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.456979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 630/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:34.462871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.561723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908957af-88e0-4d6c-b8cb-a85567deba78 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.596319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.597200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.597545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.597886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.609190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.610674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:34.625236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:34.626467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:34.627410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:34.627410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:34.627410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:34.627410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 639/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:34.632166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.734168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd08b5e-e0cd-4168-afce-b108dc3ebccb None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.768033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.768649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.768930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.769208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.769857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:34.770212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.771574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.783282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 631/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:34.789539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.821748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.894078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ab87f-9605-498c-8e4e-282e1eb745a4 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.927343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.927967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.928272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.928516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.934949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.936386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:37:34.947540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:34.948601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:34.949711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 640/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:34.954617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.012434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.056912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004479-4b20-409f-b6d0-b7587dfbb3a1 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.091244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eb53f-2edc-448a-a9ab-5054fddf4e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.091856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eb53f-2edc-448a-a9ab-5054fddf4e8a None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.092138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eb53f-2edc-448a-a9ab-5054fddf4e8a None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.092414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eb53f-2edc-448a-a9ab-5054fddf4e8a None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.093366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87eb53f-2edc-448a-a9ab-5054fddf4e8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:35.093671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87eb53f-2edc-448a-a9ab-5054fddf4e8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.095173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87eb53f-2edc-448a-a9ab-5054fddf4e8a None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.107855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 632/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.113954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.220276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.230920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5077f6-6d4c-4347-a7f3-66e3f69fa70a None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.266481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243b076-d6b9-4119-bdf8-42f8b62dbce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.267056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243b076-d6b9-4119-bdf8-42f8b62dbce0 None admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.267331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243b076-d6b9-4119-bdf8-42f8b62dbce0 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.267603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243b076-d6b9-4119-bdf8-42f8b62dbce0 None admin] PATH_INFO: `/v3/projects/e708e964ac564595b5abab68cbea89e2/users/645282ae0ac7450dbd27d71c467b5216/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.269488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243b076-d6b9-4119-bdf8-42f8b62dbce0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e708e964ac564595b5abab68cbea89e2, user_id=645282ae0ac7450dbd27d71c467b5216, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.271606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243b076-d6b9-4119-bdf8-42f8b62dbce0 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.440638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 641/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:35 2022] PUT /identity/v3/projects/e708e964ac564595b5abab68cbea89e2/users/645282ae0ac7450dbd27d71c467b5216/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:37.631554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9bd95-e770-404b-b8dd-969221aa087d None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.632079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9bd95-e770-404b-b8dd-969221aa087d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.632079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9bd95-e770-404b-b8dd-969221aa087d None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.635729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 633/1274] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:37.638900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e2edd-5806-47e0-a023-b6bb61b65a24 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.639273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e2edd-5806-47e0-a023-b6bb61b65a24 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.639441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e2edd-5806-47e0-a023-b6bb61b65a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.657601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c28e2edd-5806-47e0-a023-b6bb61b65a24 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:37.660417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c28e2edd-5806-47e0-a023-b6bb61b65a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:37.724663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 642/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:38.002607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9bd95-e770-404b-b8dd-969221aa087d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.102011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9bd95-e770-404b-b8dd-969221aa087d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.118732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.134556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0b820-ed86-4209-af24-4a4e1a2130d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.135175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0b820-ed86-4209-af24-4a4e1a2130d8 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.135450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0b820-ed86-4209-af24-4a4e1a2130d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.135762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0b820-ed86-4209-af24-4a4e1a2130d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.145840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0b820-ed86-4209-af24-4a4e1a2130d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-b5f0b820-ed86-4209-af24-4a4e1a2130d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:38.150939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:38.152078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:38.153148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 634/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:38.158267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e2edd-5806-47e0-a023-b6bb61b65a24 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.247096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.266988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e2edd-5806-47e0-a023-b6bb61b65a24 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.301288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.301914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.302225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.302505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.303169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:38.303474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.304838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.313270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 643/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:38.319827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0b820-ed86-4209-af24-4a4e1a2130d8 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.417591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0b820-ed86-4209-af24-4a4e1a2130d8 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.450702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.451324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.451608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.451884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.462217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.466264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:38.478239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:38.479960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:38.481784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:38.481784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:38.481784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:38.481784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 635/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:38.485349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.584594 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8917d-92d3-48a6-af0b-b73a1598a368 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.618950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.619558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.619839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.620116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.620753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:38.621056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.622453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.635070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 644/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:38.640976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.671045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.739918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1b501-eb2b-43d1-bf5e-9a9eb18d407b None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.773990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.774645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.774928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.775203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] PATH_INFO: `/v3/users/placement` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.782084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.784495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:37:38.795226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:38.797065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:38.798875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 636/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:38.803031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.807097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.904348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8dad6-d627-4f3b-94aa-400f88a951ae None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.938800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.939410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.939691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.939964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.940691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:38.940998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.943043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.959813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 645/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:38.965691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.054004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.063389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98933a70-6a0d-4531-9deb-766a9e7e9cfc None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.095439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fe53e-e7d9-40ed-b15e-10f7a8b3ae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.096081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe53e-e7d9-40ed-b15e-10f7a8b3ae40 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.096329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe53e-e7d9-40ed-b15e-10f7a8b3ae40 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.096587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe53e-e7d9-40ed-b15e-10f7a8b3ae40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.164047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fe53e-e7d9-40ed-b15e-10f7a8b3ae40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.165463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fe53e-e7d9-40ed-b15e-10f7a8b3ae40 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.270363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.280614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 637/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:37:39.286975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.352781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.391478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e0b66-4428-4bd1-9059-e0a51e7d0cca None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.426630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.427197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.427478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.427753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.437682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.439299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:37:39.453895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:39.455121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:37:39.456129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:37:39.456129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:37:39.456129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:39.456129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 646/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:39.461653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fe53e-e7d9-40ed-b15e-10f7a8b3ae40 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.560516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fe53e-e7d9-40ed-b15e-10f7a8b3ae40 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.597110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.597716 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.597995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.598307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.599526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:39.599830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.601192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.612871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 638/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:39.618664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.720438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870326ea-540b-4bd8-9c1c-6386868066fd None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.734778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.754126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.755308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.755610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.755877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.762481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.763917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:37:39.776784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:39.777771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: ERROR keystone.server.flask.application Sep 08 13:37:39.778757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 647/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:39.783881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.832611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.882919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd26e4d-ede4-46a5-aed7-004a624cc293 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.916414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409841f5-d1e3-4999-8492-885f198cc1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.917079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409841f5-d1e3-4999-8492-885f198cc1ab None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.917363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409841f5-d1e3-4999-8492-885f198cc1ab None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.917643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409841f5-d1e3-4999-8492-885f198cc1ab None admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.918365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409841f5-d1e3-4999-8492-885f198cc1ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:39.918674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409841f5-d1e3-4999-8492-885f198cc1ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.920108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409841f5-d1e3-4999-8492-885f198cc1ab None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.932545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 639/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:39.938844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:40.044373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f941a5-407e-4781-a885-b6821ab82594 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:40.077059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06dd61-3ae1-45d6-a005-2d26090d64b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:40.077650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06dd61-3ae1-45d6-a005-2d26090d64b6 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:40.077956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06dd61-3ae1-45d6-a005-2d26090d64b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:40.078326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06dd61-3ae1-45d6-a005-2d26090d64b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:40.079048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06dd61-3ae1-45d6-a005-2d26090d64b6 None admin] RBAC: Adding query filter params (role.id=97f11873550e41dba0cd095afb1dec00, scope.project.id=e708e964ac564595b5abab68cbea89e2, user.id=645282ae0ac7450dbd27d71c467b5216) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:40.079356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06dd61-3ae1-45d6-a005-2d26090d64b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:40.080775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06dd61-3ae1-45d6-a005-2d26090d64b6 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:40.086975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 648/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/role_assignments?user.id=645282ae0ac7450dbd27d71c467b5216&scope.project.id=e708e964ac564595b5abab68cbea89e2&role.id=97f11873550e41dba0cd095afb1dec00 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:42.306319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961233-19d3-422a-b4fc-e8684bf0ac21 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.307107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961233-19d3-422a-b4fc-e8684bf0ac21 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.307107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961233-19d3-422a-b4fc-e8684bf0ac21 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.308234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 640/1288] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:42.314235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a9dee-90c3-4b44-9b76-ddb763da2c1c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.315397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a9dee-90c3-4b44-9b76-ddb763da2c1c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.315397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a9dee-90c3-4b44-9b76-ddb763da2c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.335608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f69a9dee-90c3-4b44-9b76-ddb763da2c1c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:42.336852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f69a9dee-90c3-4b44-9b76-ddb763da2c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:42.373036 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:42.420102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 649/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:42.697996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58961233-19d3-422a-b4fc-e8684bf0ac21 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:42.797069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58961233-19d3-422a-b4fc-e8684bf0ac21 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:42.808141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:42.831811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:42.832454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.832775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.833632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] PATH_INFO: `/v3/services/placement` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.834248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:42.835611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:42.845650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:42.846969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:42.847909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 641/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:37:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:42.852998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a9dee-90c3-4b44-9b76-ddb763da2c1c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:42.949811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a9dee-90c3-4b44-9b76-ddb763da2c1c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:42.983455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19b96e-f17e-4014-aa4f-c8b935d2d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:42.984058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19b96e-f17e-4014-aa4f-c8b935d2d655 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.989329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19b96e-f17e-4014-aa4f-c8b935d2d655 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.989329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19b96e-f17e-4014-aa4f-c8b935d2d655 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.989329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19b96e-f17e-4014-aa4f-c8b935d2d655 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:42.989329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19b96e-f17e-4014-aa4f-c8b935d2d655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:42.989329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19b96e-f17e-4014-aa4f-c8b935d2d655 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:42.991621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 650/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 8 13:37:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:42.997488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.096104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea5b0-247e-49fa-a02b-59a21a5da94c None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.128341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a1621-9f5d-4a9e-92dc-cc0095b99a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.128938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1621-9f5d-4a9e-92dc-cc0095b99a70 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.129217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1621-9f5d-4a9e-92dc-cc0095b99a70 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.129490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1621-9f5d-4a9e-92dc-cc0095b99a70 None admin] PATH_INFO: `/v3/services` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.130691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a1621-9f5d-4a9e-92dc-cc0095b99a70 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:43.130995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a1621-9f5d-4a9e-92dc-cc0095b99a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.132352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a1621-9f5d-4a9e-92dc-cc0095b99a70 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.136636 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 642/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 8 13:37:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:45.283013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d82aba-5574-4544-92c3-4741ca25f569 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.283854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d82aba-5574-4544-92c3-4741ca25f569 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.283854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d82aba-5574-4544-92c3-4741ca25f569 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.285453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 651/1293] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:45.291238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca413-ffbe-42c9-abe9-928c38019bdd None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.291635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca413-ffbe-42c9-abe9-928c38019bdd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.291799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca413-ffbe-42c9-abe9-928c38019bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.313682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-bdbca413-ffbe-42c9-abe9-928c38019bdd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:45.314948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bdbca413-ffbe-42c9-abe9-928c38019bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:45.349435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.386766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 643/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:45.661551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d82aba-5574-4544-92c3-4741ca25f569 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.695431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.777593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d82aba-5574-4544-92c3-4741ca25f569 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.812651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e76ec-fa70-4546-89b1-948a5c1da28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.813268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e76ec-fa70-4546-89b1-948a5c1da28c None admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.813551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e76ec-fa70-4546-89b1-948a5c1da28c None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.813828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e76ec-fa70-4546-89b1-948a5c1da28c None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.814593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e76ec-fa70-4546-89b1-948a5c1da28c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.815969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e76ec-fa70-4546-89b1-948a5c1da28c None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.898645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 652/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 13:37:45 2022] POST /identity/v3/services => generated 246 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:48.081584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e21f93-acb7-413c-96b8-31e619b0a13a None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.082119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e21f93-acb7-413c-96b8-31e619b0a13a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.082238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e21f93-acb7-413c-96b8-31e619b0a13a None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.083580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 644/1296] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:48.088820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f1f86-0162-401a-aba6-2e1a6830fd14 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.089274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f1f86-0162-401a-aba6-2e1a6830fd14 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.089391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f1f86-0162-401a-aba6-2e1a6830fd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.108638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0b0f1f86-0162-401a-aba6-2e1a6830fd14 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:48.109861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0b0f1f86-0162-401a-aba6-2e1a6830fd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:48.187705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 653/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:48.455436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e21f93-acb7-413c-96b8-31e619b0a13a None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.561323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e21f93-acb7-413c-96b8-31e619b0a13a None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.599395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.600032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.600315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.600591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.601186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.602572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:48.611512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:48.612754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:48.613849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 645/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:48.618688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f1f86-0162-401a-aba6-2e1a6830fd14 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.637323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.720661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f1f86-0162-401a-aba6-2e1a6830fd14 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.755948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.756547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.756820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.757089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.757709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:48.758005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.759441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.764447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 654/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.771123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.802026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.873485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f2123-fc4e-4d73-840d-dc1e69f94103 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.907403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db91be-84c5-4091-a8e0-2ec598ec36fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.908011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db91be-84c5-4091-a8e0-2ec598ec36fc None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.908345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db91be-84c5-4091-a8e0-2ec598ec36fc None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.908625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db91be-84c5-4091-a8e0-2ec598ec36fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.909324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db91be-84c5-4091-a8e0-2ec598ec36fc None admin] RBAC: Adding query filter params (interface=public, service_id=edcb4f8435874c02b49eb6f6996045aa, region_id=RegionOne) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:48.909631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db91be-84c5-4091-a8e0-2ec598ec36fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.911039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db91be-84c5-4091-a8e0-2ec598ec36fc None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.916158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 646/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/endpoints?service_id=edcb4f8435874c02b49eb6f6996045aa&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.922156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.011216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:49.028787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cf911-e639-4c7a-912c-8842b80b28f7 None admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.061461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50f844-135e-44af-ab65-3bdc7117aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.062058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50f844-135e-44af-ab65-3bdc7117aa80 None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.062360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50f844-135e-44af-ab65-3bdc7117aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.062731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50f844-135e-44af-ab65-3bdc7117aa80 None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.063286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50f844-135e-44af-ab65-3bdc7117aa80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.064630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50f844-135e-44af-ab65-3bdc7117aa80 None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.072199 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 655/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:37:51.251268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eef55-2aa5-493f-a088-4008f86d6084 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.251803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eef55-2aa5-493f-a088-4008f86d6084 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.251803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eef55-2aa5-493f-a088-4008f86d6084 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.253161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 647/1302] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:51.258550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c8275-af0b-428e-9515-59fbdb0bdc10 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.258986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c8275-af0b-428e-9515-59fbdb0bdc10 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.259098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c8275-af0b-428e-9515-59fbdb0bdc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.277195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-070c8275-af0b-428e-9515-59fbdb0bdc10 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:51.278541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-070c8275-af0b-428e-9515-59fbdb0bdc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:51.350313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 656/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:37:51.641624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eef55-2aa5-493f-a088-4008f86d6084 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.738823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:51.746697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eef55-2aa5-493f-a088-4008f86d6084 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.783476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:51.784114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.784387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.784657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.785237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:51.787112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:51.795698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:51.797000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: ERROR keystone.server.flask.application Sep 08 13:37:51.797918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 648/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:51.804021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c8275-af0b-428e-9515-59fbdb0bdc10 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.882244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:51.904337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c8275-af0b-428e-9515-59fbdb0bdc10 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.937857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf0991-6680-463f-9aa1-0494a77072cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:51.938574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf0991-6680-463f-9aa1-0494a77072cd None admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.939029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf0991-6680-463f-9aa1-0494a77072cd None admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.939503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf0991-6680-463f-9aa1-0494a77072cd None admin] PATH_INFO: `/v3/services` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.940162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf0991-6680-463f-9aa1-0494a77072cd None admin] RBAC: Adding query filter params (name=placement) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:51.940460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf0991-6680-463f-9aa1-0494a77072cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:51.941818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf0991-6680-463f-9aa1-0494a77072cd None admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:51.947130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 657/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:51.952929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:52.057148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c039af1-5cfa-4c43-b3a5-b601611fb710 None admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.091276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210e5bf-448f-4239-96c2-fd7b1f7d000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.091945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210e5bf-448f-4239-96c2-fd7b1f7d000c None admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.092228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210e5bf-448f-4239-96c2-fd7b1f7d000c None admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.092500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210e5bf-448f-4239-96c2-fd7b1f7d000c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.093236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210e5bf-448f-4239-96c2-fd7b1f7d000c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:52.094844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210e5bf-448f-4239-96c2-fd7b1f7d000c None admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:52.223061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 649/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:37:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:01.944075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=30 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.217651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:01.863724 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6406f-a112-4551-af77-cc4887986c19 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:01.864384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6406f-a112-4551-af77-cc4887986c19 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:01.864384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6406f-a112-4551-af77-cc4887986c19 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:01.865765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 658/1307] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:01.871074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff53c-5029-4fac-88eb-c5b75fd4f472 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:01.871547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff53c-5029-4fac-88eb-c5b75fd4f472 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:01.871632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff53c-5029-4fac-88eb-c5b75fd4f472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:02.002789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9a0ff53c-5029-4fac-88eb-c5b75fd4f472 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:02.004471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9a0ff53c-5029-4fac-88eb-c5b75fd4f472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:02.140286 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 650/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:02.439986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d2e47-7790-4d5a-8ee7-9dff5cd91cee None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:02.440268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d2e47-7790-4d5a-8ee7-9dff5cd91cee None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:02.440702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d2e47-7790-4d5a-8ee7-9dff5cd91cee None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:02.443787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 659/1309] 10.222.0.25 () {54 vars in 965 bytes} [Thu Sep 8 13:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:02.450509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc6604-5870-46db-9b02-d12f216ee9fb None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:02.450872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc6604-5870-46db-9b02-d12f216ee9fb None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:02.451223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc6604-5870-46db-9b02-d12f216ee9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:02.545173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6ddc6604-5870-46db-9b02-d12f216ee9fb None None] MFA Rules not processed for user `a71d353c1bd849028930261e9306fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:02.546590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6ddc6604-5870-46db-9b02-d12f216ee9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:02.570941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:02.645245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 651/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Sep 8 13:45:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:02.654701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d2e47-7790-4d5a-8ee7-9dff5cd91cee None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:02.657139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-197d2e47-7790-4d5a-8ee7-9dff5cd91cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:02.796050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d2e47-7790-4d5a-8ee7-9dff5cd91cee None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:02.883077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5dfbf-2bec-4d62-8b0d-1941760e7556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:02.883720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5dfbf-2bec-4d62-8b0d-1941760e7556 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:02.883971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5dfbf-2bec-4d62-8b0d-1941760e7556 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:02.884237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5dfbf-2bec-4d62-8b0d-1941760e7556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:02.885851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e5a5dfbf-2bec-4d62-8b0d-1941760e7556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:02.956298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:03.029361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5dfbf-2bec-4d62-8b0d-1941760e7556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:03.031077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5dfbf-2bec-4d62-8b0d-1941760e7556 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:03.057071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 660/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:05.627116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e22d32-b024-482e-bbf2-b50be232f346 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:05.627746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e22d32-b024-482e-bbf2-b50be232f346 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:05.627746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e22d32-b024-482e-bbf2-b50be232f346 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:05.629059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 652/1312] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:05.634585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef600-a390-446c-9a50-19864d904489 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:05.635387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef600-a390-446c-9a50-19864d904489 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:05.635387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef600-a390-446c-9a50-19864d904489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:05.711568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d6cef600-a390-446c-9a50-19864d904489 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:05.712908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d6cef600-a390-446c-9a50-19864d904489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:05.815211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 661/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:06.097612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e22d32-b024-482e-bbf2-b50be232f346 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:06.099183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-68e22d32-b024-482e-bbf2-b50be232f346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:06.204403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e22d32-b024-482e-bbf2-b50be232f346 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:06.280445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:06.287072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb114c-6dc8-4890-b9e9-b11f78dd5a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:06.288402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb114c-6dc8-4890-b9e9-b11f78dd5a11 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:06.292056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb114c-6dc8-4890-b9e9-b11f78dd5a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:06.292056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb114c-6dc8-4890-b9e9-b11f78dd5a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:06.292056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fddb114c-6dc8-4890-b9e9-b11f78dd5a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:06.401466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb114c-6dc8-4890-b9e9-b11f78dd5a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:06.402961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb114c-6dc8-4890-b9e9-b11f78dd5a11 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:06.421619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 653/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:09.117670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345d54c-f0ff-407e-9702-979ddbef505b None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:09.118301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345d54c-f0ff-407e-9702-979ddbef505b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:09.118301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345d54c-f0ff-407e-9702-979ddbef505b None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:09.119646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 662/1315] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:09.125229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93807fe4-0f71-4404-b1ea-1fa1965a0478 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:09.126247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93807fe4-0f71-4404-b1ea-1fa1965a0478 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:09.126358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93807fe4-0f71-4404-b1ea-1fa1965a0478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:09.193661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-93807fe4-0f71-4404-b1ea-1fa1965a0478 None None] MFA Rules not processed for user `0a20f1b54d6d470882694c7186cd6fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:09.194953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-93807fe4-0f71-4404-b1ea-1fa1965a0478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:09.249311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:09.294524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 654/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:09 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:09.602006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345d54c-f0ff-407e-9702-979ddbef505b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:09.603615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5345d54c-f0ff-407e-9702-979ddbef505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:09.614896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:09.721301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345d54c-f0ff-407e-9702-979ddbef505b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:09.797020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188345a0-5613-493c-8ded-248c27b077be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:09.797627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188345a0-5613-493c-8ded-248c27b077be service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:09.797906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188345a0-5613-493c-8ded-248c27b077be service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:09.798203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188345a0-5613-493c-8ded-248c27b077be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:09.799793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-188345a0-5613-493c-8ded-248c27b077be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:09.863987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:09.910888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188345a0-5613-493c-8ded-248c27b077be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:09.912343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188345a0-5613-493c-8ded-248c27b077be service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:09.935855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 663/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:13.094260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45d35b-9974-47d7-9307-3774a0a2c6f2 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:13.094826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45d35b-9974-47d7-9307-3774a0a2c6f2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:13.094826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45d35b-9974-47d7-9307-3774a0a2c6f2 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:13.096698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 655/1318] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:13.102119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd728d-08e6-4267-bc5b-6e80740989f1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:13.102636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd728d-08e6-4267-bc5b-6e80740989f1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:13.115678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd728d-08e6-4267-bc5b-6e80740989f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:13.189776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-cfdd728d-08e6-4267-bc5b-6e80740989f1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:13.191406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cfdd728d-08e6-4267-bc5b-6e80740989f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:13.302720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 664/1319] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:13.586167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45d35b-9974-47d7-9307-3774a0a2c6f2 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:13.587724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6c45d35b-9974-47d7-9307-3774a0a2c6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:13.710792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45d35b-9974-47d7-9307-3774a0a2c6f2 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:13.787569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94ec90-6353-4fa3-b4a1-f5c69c646cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:13.788185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ec90-6353-4fa3-b4a1-f5c69c646cdc service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:13.788469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ec90-6353-4fa3-b4a1-f5c69c646cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:13.788744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ec90-6353-4fa3-b4a1-f5c69c646cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:13.790371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7b94ec90-6353-4fa3-b4a1-f5c69c646cdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:13.797486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:13.899355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94ec90-6353-4fa3-b4a1-f5c69c646cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:13.900794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94ec90-6353-4fa3-b4a1-f5c69c646cdc service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:13.919329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 656/1320] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:17.243719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b03368-1218-4ddf-a846-82873952bf57 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:17.244300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b03368-1218-4ddf-a846-82873952bf57 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:17.244300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b03368-1218-4ddf-a846-82873952bf57 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:17.245656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 665/1321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:17.252408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d1d87-55be-4fef-b9bf-5d44a26fa434 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:17.255164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d1d87-55be-4fef-b9bf-5d44a26fa434 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:17.255164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d1d87-55be-4fef-b9bf-5d44a26fa434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:17.330279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-642d1d87-55be-4fef-b9bf-5d44a26fa434 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:17.331986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-642d1d87-55be-4fef-b9bf-5d44a26fa434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:17.409556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:17.450643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 657/1322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:17.748093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b03368-1218-4ddf-a846-82873952bf57 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:17.749866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-35b03368-1218-4ddf-a846-82873952bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:17.771247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:17.864854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b03368-1218-4ddf-a846-82873952bf57 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:17.965372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe9d16-68ea-4b0c-9551-bf86b17df0e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:17.965619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe9d16-68ea-4b0c-9551-bf86b17df0e3 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:17.965923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe9d16-68ea-4b0c-9551-bf86b17df0e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:17.966497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe9d16-68ea-4b0c-9551-bf86b17df0e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:17.969345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6cfe9d16-68ea-4b0c-9551-bf86b17df0e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:18.058593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:18.117630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe9d16-68ea-4b0c-9551-bf86b17df0e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:18.119110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe9d16-68ea-4b0c-9551-bf86b17df0e3 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:18.138998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 666/1323] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:20.381130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce8568-75cb-45f6-b9fb-7764e6d254af None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:20.381687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce8568-75cb-45f6-b9fb-7764e6d254af None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:20.381687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce8568-75cb-45f6-b9fb-7764e6d254af None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:20.383063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 658/1324] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:20.388438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3615e-b3c2-45d9-b947-2586bdabffd2 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:20.388920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3615e-b3c2-45d9-b947-2586bdabffd2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:20.389004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3615e-b3c2-45d9-b947-2586bdabffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:20.473821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-24c3615e-b3c2-45d9-b947-2586bdabffd2 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:20.475073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-24c3615e-b3c2-45d9-b947-2586bdabffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:20.572974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 667/1325] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:20.867311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731d705-48a3-4722-96b8-7993c8236474 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:20.867603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731d705-48a3-4722-96b8-7993c8236474 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:20.868047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731d705-48a3-4722-96b8-7993c8236474 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:20.869241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 659/1326] 10.222.0.25 () {54 vars in 965 bytes} [Thu Sep 8 13:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:20.875202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e922-cce5-4e38-a39f-51cdf3e90534 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:20.875487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e922-cce5-4e38-a39f-51cdf3e90534 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:20.875784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e922-cce5-4e38-a39f-51cdf3e90534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:20.909020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:20.956215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-99d6e922-cce5-4e38-a39f-51cdf3e90534 None None] MFA Rules not processed for user `a71d353c1bd849028930261e9306fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:20.957420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-99d6e922-cce5-4e38-a39f-51cdf3e90534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:21.035931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 668/1327] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Sep 8 13:45:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:21.044962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731d705-48a3-4722-96b8-7993c8236474 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:21.046514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0731d705-48a3-4722-96b8-7993c8236474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:21.151125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731d705-48a3-4722-96b8-7993c8236474 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:21.225257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be86065-ab6d-4d5c-8669-35dc8eb667fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:21.225863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86065-ab6d-4d5c-8669-35dc8eb667fd service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:21.226170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86065-ab6d-4d5c-8669-35dc8eb667fd service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:21.226511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86065-ab6d-4d5c-8669-35dc8eb667fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:21.228059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3be86065-ab6d-4d5c-8669-35dc8eb667fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:21.245343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:21.344268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be86065-ab6d-4d5c-8669-35dc8eb667fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:21.346237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be86065-ab6d-4d5c-8669-35dc8eb667fd service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:21.366112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 660/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:23.860253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd145d-6a90-45eb-b520-807ef15b0f41 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:23.861574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd145d-6a90-45eb-b520-807ef15b0f41 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:23.861574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd145d-6a90-45eb-b520-807ef15b0f41 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:23.863005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 669/1329] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:23.868532 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df348eaf-d2b8-442a-b7d7-12181b85df86 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:23.868922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df348eaf-d2b8-442a-b7d7-12181b85df86 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:23.869022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df348eaf-d2b8-442a-b7d7-12181b85df86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:23.941766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-df348eaf-d2b8-442a-b7d7-12181b85df86 None None] MFA Rules not processed for user `0a20f1b54d6d470882694c7186cd6fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:23.943043 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-df348eaf-d2b8-442a-b7d7-12181b85df86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:24.017596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:24.053105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 661/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:24.382424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd145d-6a90-45eb-b520-807ef15b0f41 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:24.383962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e7fd145d-6a90-45eb-b520-807ef15b0f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:24.496962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd145d-6a90-45eb-b520-807ef15b0f41 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:24.518611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:24.575363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0d45b-a29a-4762-8679-4415ca778446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:24.575973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d45b-a29a-4762-8679-4415ca778446 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:24.576257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d45b-a29a-4762-8679-4415ca778446 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:24.576531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d45b-a29a-4762-8679-4415ca778446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:24.578130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-90a0d45b-a29a-4762-8679-4415ca778446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:24.699374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0d45b-a29a-4762-8679-4415ca778446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:24.701001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0d45b-a29a-4762-8679-4415ca778446 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:24.720420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 670/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:27.390738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f128-96ea-411c-8d4c-1fb0ea209948 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:27.391305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f128-96ea-411c-8d4c-1fb0ea209948 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:27.391305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f128-96ea-411c-8d4c-1fb0ea209948 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:27.393180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 662/1332] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:27.398529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9a62a-7e69-483e-8ef3-308a4ea3df0f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:27.398975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9a62a-7e69-483e-8ef3-308a4ea3df0f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:27.399072 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9a62a-7e69-483e-8ef3-308a4ea3df0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:27.463922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:27.467057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a1b9a62a-7e69-483e-8ef3-308a4ea3df0f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:27.468308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a1b9a62a-7e69-483e-8ef3-308a4ea3df0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:27.580769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 671/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:27.870443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471f128-96ea-411c-8d4c-1fb0ea209948 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:27.872085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1471f128-96ea-411c-8d4c-1fb0ea209948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:27.976099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471f128-96ea-411c-8d4c-1fb0ea209948 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:28.056945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d6773-c6f9-4a72-aadd-db509342cc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:28.057558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d6773-c6f9-4a72-aadd-db509342cc57 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:28.057839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d6773-c6f9-4a72-aadd-db509342cc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:28.058116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d6773-c6f9-4a72-aadd-db509342cc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:28.059749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-540d6773-c6f9-4a72-aadd-db509342cc57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:28.087968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:28.170807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d6773-c6f9-4a72-aadd-db509342cc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:28.172180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d6773-c6f9-4a72-aadd-db509342cc57 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:28.191539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 663/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:30.409744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f99c9-a58a-46c2-825e-ee30300d4416 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:30.410363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f99c9-a58a-46c2-825e-ee30300d4416 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:30.410363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f99c9-a58a-46c2-825e-ee30300d4416 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:30.411658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 672/1335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:30.417333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8904b0-8dfd-45d8-91de-e1173c06203f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:30.417738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8904b0-8dfd-45d8-91de-e1173c06203f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:30.417888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8904b0-8dfd-45d8-91de-e1173c06203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:30.487106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9e8904b0-8dfd-45d8-91de-e1173c06203f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:30.488325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9e8904b0-8dfd-45d8-91de-e1173c06203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:30.567498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:30.593220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 664/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:30.882653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f99c9-a58a-46c2-825e-ee30300d4416 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:30.884214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-681f99c9-a58a-46c2-825e-ee30300d4416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:30.991600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f99c9-a58a-46c2-825e-ee30300d4416 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:31.020302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:31.067285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf8dac-17ce-463a-9366-53d77026a7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:31.067809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf8dac-17ce-463a-9366-53d77026a7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:31.068092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf8dac-17ce-463a-9366-53d77026a7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:31.068367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf8dac-17ce-463a-9366-53d77026a7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:31.069977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6ddf8dac-17ce-463a-9366-53d77026a7c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:31.188192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf8dac-17ce-463a-9366-53d77026a7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:31.189741 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf8dac-17ce-463a-9366-53d77026a7c4 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:31.207830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 673/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:33.963392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988dbe7-ea6f-48cd-be46-2588ebb60bae None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:33.963910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988dbe7-ea6f-48cd-be46-2588ebb60bae None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:33.964006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988dbe7-ea6f-48cd-be46-2588ebb60bae None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:33.965303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 665/1338] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:33.970624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035c2be-a583-441e-be2e-246ae0431e2d None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:33.971071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035c2be-a583-441e-be2e-246ae0431e2d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:33.971176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035c2be-a583-441e-be2e-246ae0431e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:34.041684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e035c2be-a583-441e-be2e-246ae0431e2d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:34.042937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e035c2be-a583-441e-be2e-246ae0431e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:34.051203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:34.144014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 674/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:34.431580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988dbe7-ea6f-48cd-be46-2588ebb60bae None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:34.433130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0988dbe7-ea6f-48cd-be46-2588ebb60bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:34.540216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988dbe7-ea6f-48cd-be46-2588ebb60bae None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:34.616438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576162c8-a13c-450b-8df7-90a2e561cbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:34.617092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576162c8-a13c-450b-8df7-90a2e561cbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:34.617391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576162c8-a13c-450b-8df7-90a2e561cbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:34.617391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576162c8-a13c-450b-8df7-90a2e561cbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:34.617391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-576162c8-a13c-450b-8df7-90a2e561cbf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:34.655879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:34.728974 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576162c8-a13c-450b-8df7-90a2e561cbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:34.730356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576162c8-a13c-450b-8df7-90a2e561cbf5 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:34.749649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 666/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:39.653968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f32d69-2606-4e46-b58e-2c5e7a411b24 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:39.654611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f32d69-2606-4e46-b58e-2c5e7a411b24 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:39.654611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f32d69-2606-4e46-b58e-2c5e7a411b24 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:39.656358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 675/1341] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:39.662676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c82b-027b-43ba-9d2b-4e2e2eea6217 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:39.663185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c82b-027b-43ba-9d2b-4e2e2eea6217 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:39.663327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c82b-027b-43ba-9d2b-4e2e2eea6217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:39.732343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-59c9c82b-027b-43ba-9d2b-4e2e2eea6217 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:39.733570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-59c9c82b-027b-43ba-9d2b-4e2e2eea6217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:39.820234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:39.831887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 667/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:40.121808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f32d69-2606-4e46-b58e-2c5e7a411b24 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:40.123501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a0f32d69-2606-4e46-b58e-2c5e7a411b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:40.237443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f32d69-2606-4e46-b58e-2c5e7a411b24 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:40.277393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:40.314477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330dcaee-3ace-4182-a326-45e1a379eae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:40.315236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcaee-3ace-4182-a326-45e1a379eae9 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:40.315521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcaee-3ace-4182-a326-45e1a379eae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:40.315791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcaee-3ace-4182-a326-45e1a379eae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:40.317450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-330dcaee-3ace-4182-a326-45e1a379eae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:40.428884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dcaee-3ace-4182-a326-45e1a379eae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:40.430284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dcaee-3ace-4182-a326-45e1a379eae9 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:40.449781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 676/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:43.925454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8d6f3-1dbf-4b2c-a355-e540dbff4ce8 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:43.926014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8d6f3-1dbf-4b2c-a355-e540dbff4ce8 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:43.926014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8d6f3-1dbf-4b2c-a355-e540dbff4ce8 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:43.927406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 668/1344] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:43.932734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9abb2-d4d5-4758-8bd4-db58f604489a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:43.933177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9abb2-d4d5-4758-8bd4-db58f604489a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:43.933261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9abb2-d4d5-4758-8bd4-db58f604489a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:44.003359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-70f9abb2-d4d5-4758-8bd4-db58f604489a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:44.004541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-70f9abb2-d4d5-4758-8bd4-db58f604489a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:44.025363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:44.109962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 677/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:44.392963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8d6f3-1dbf-4b2c-a355-e540dbff4ce8 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:44.394557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4bf8d6f3-1dbf-4b2c-a355-e540dbff4ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:44.526002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8d6f3-1dbf-4b2c-a355-e540dbff4ce8 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:44.613953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cfd79-d35a-4a54-a0b6-56dfe9604841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:44.614224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cfd79-d35a-4a54-a0b6-56dfe9604841 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:44.614910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cfd79-d35a-4a54-a0b6-56dfe9604841 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:44.614910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cfd79-d35a-4a54-a0b6-56dfe9604841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:44.618110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-018cfd79-d35a-4a54-a0b6-56dfe9604841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:44.675740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:44.756990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018cfd79-d35a-4a54-a0b6-56dfe9604841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:44.758380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018cfd79-d35a-4a54-a0b6-56dfe9604841 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:44.779147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 669/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:49.216278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d645d7-1bec-4e8f-9079-b89b867646b0 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:49.216819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d645d7-1bec-4e8f-9079-b89b867646b0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:49.216819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d645d7-1bec-4e8f-9079-b89b867646b0 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:49.218281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 678/1347] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:49.223929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f158b-2b85-44af-922f-1102733f775d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:49.224454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f158b-2b85-44af-922f-1102733f775d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:49.224554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f158b-2b85-44af-922f-1102733f775d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:49.299546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-680f158b-2b85-44af-922f-1102733f775d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:49.300928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-680f158b-2b85-44af-922f-1102733f775d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:49.414586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:49.416931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 670/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:49.704772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d645d7-1bec-4e8f-9079-b89b867646b0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:49.706401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-90d645d7-1bec-4e8f-9079-b89b867646b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:49.820251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d645d7-1bec-4e8f-9079-b89b867646b0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:49.873401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:49.899683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0731-cf7a-43e5-b6d2-982a62b61ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:49.900288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0731-cf7a-43e5-b6d2-982a62b61ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:49.900570 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0731-cf7a-43e5-b6d2-982a62b61ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:49.900846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0731-cf7a-43e5-b6d2-982a62b61ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:49.902479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-11de0731-cf7a-43e5-b6d2-982a62b61ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:50.031965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de0731-cf7a-43e5-b6d2-982a62b61ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:50.033410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de0731-cf7a-43e5-b6d2-982a62b61ef5 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:50.052687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 679/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:45:55.407607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aec29a-2476-435f-9684-46219ae8cb06 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:55.408111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aec29a-2476-435f-9684-46219ae8cb06 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:55.408111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aec29a-2476-435f-9684-46219ae8cb06 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:55.409773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 671/1350] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:55.415202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3d089-c27e-44bd-9547-30a5cd293334 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:55.415606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3d089-c27e-44bd-9547-30a5cd293334 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:55.415755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3d089-c27e-44bd-9547-30a5cd293334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:55.526307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-3ee3d089-c27e-44bd-9547-30a5cd293334 None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:55.528060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3ee3d089-c27e-44bd-9547-30a5cd293334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:55.550746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:55.747763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 680/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:45:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:00.340580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ad6b-a4f1-402c-9486-02ec2e511c09 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:00.340580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ad6b-a4f1-402c-9486-02ec2e511c09 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:00.340580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ad6b-a4f1-402c-9486-02ec2e511c09 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:00.342187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 672/1352] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:00.348943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da4d1f-9cc6-4f2e-a879-66eb02f28715 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:00.349454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da4d1f-9cc6-4f2e-a879-66eb02f28715 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:00.349637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da4d1f-9cc6-4f2e-a879-66eb02f28715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:00.487739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-69da4d1f-9cc6-4f2e-a879-66eb02f28715 None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:00.489860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-69da4d1f-9cc6-4f2e-a879-66eb02f28715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:00.604361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:00.658177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 681/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:46:01.858837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdbe9-c73c-4a38-b003-36a5dd5ba668 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:01.859356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdbe9-c73c-4a38-b003-36a5dd5ba668 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:01.860067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdbe9-c73c-4a38-b003-36a5dd5ba668 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:01.861574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 673/1354] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:01.867476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b1d5d-5b1f-497d-9ac4-2b0bdc899de4 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:01.868000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b1d5d-5b1f-497d-9ac4-2b0bdc899de4 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:01.868000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b1d5d-5b1f-497d-9ac4-2b0bdc899de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:01.953111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f92b1d5d-5b1f-497d-9ac4-2b0bdc899de4 None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:01.954416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f92b1d5d-5b1f-497d-9ac4-2b0bdc899de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:02.071511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 682/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:46:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:05.683422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a1f0-b431-4504-9cb1-8fc7aded45f2 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:05.684388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a1f0-b431-4504-9cb1-8fc7aded45f2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:05.684868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a1f0-b431-4504-9cb1-8fc7aded45f2 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:05.686780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 674/1356] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:05.694849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705c6b-225a-49a1-95aa-2e36fd3e7b3a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:05.695789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705c6b-225a-49a1-95aa-2e36fd3e7b3a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:05.696298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705c6b-225a-49a1-95aa-2e36fd3e7b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:05.748309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:05.795245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-67705c6b-225a-49a1-95aa-2e36fd3e7b3a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:05.796739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-67705c6b-225a-49a1-95aa-2e36fd3e7b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:05.917697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 683/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:06.219915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ae6ec-a611-4975-8f57-9d76557db9d2 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:06.220603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ae6ec-a611-4975-8f57-9d76557db9d2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:06.221089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ae6ec-a611-4975-8f57-9d76557db9d2 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:06.223129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 675/1358] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 8 13:46:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:06.232295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58d35-1d5a-4952-9985-82d24d05b137 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:06.232871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58d35-1d5a-4952-9985-82d24d05b137 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:06.233457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58d35-1d5a-4952-9985-82d24d05b137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:06.329403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-74b58d35-1d5a-4952-9985-82d24d05b137 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:06.330677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-74b58d35-1d5a-4952-9985-82d24d05b137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:06.395716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:06.475460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 684/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 8 13:46:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:46:06.483099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ae6ec-a611-4975-8f57-9d76557db9d2 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:06.486274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5b4ae6ec-a611-4975-8f57-9d76557db9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:06.675925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ae6ec-a611-4975-8f57-9d76557db9d2 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:06.759307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:06.801355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd09537-8839-4bdf-b5a2-5d1e7b1beaa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:06.802130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd09537-8839-4bdf-b5a2-5d1e7b1beaa7 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:06.802495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd09537-8839-4bdf-b5a2-5d1e7b1beaa7 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:06.802771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd09537-8839-4bdf-b5a2-5d1e7b1beaa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:06.804446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0fd09537-8839-4bdf-b5a2-5d1e7b1beaa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:06.911645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd09537-8839-4bdf-b5a2-5d1e7b1beaa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:06.913230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd09537-8839-4bdf-b5a2-5d1e7b1beaa7 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:06.936002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 676/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:09.413196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8697b-88cb-4119-af5a-1ba169e63ffa None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:09.413801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8697b-88cb-4119-af5a-1ba169e63ffa None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:09.413801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8697b-88cb-4119-af5a-1ba169e63ffa None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:09.415352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 685/1361] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:09.420575 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc344df-60f9-40f7-ac1c-1de2271e63b4 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:09.421266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc344df-60f9-40f7-ac1c-1de2271e63b4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:09.421266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc344df-60f9-40f7-ac1c-1de2271e63b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:09.526305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ddc344df-60f9-40f7-ac1c-1de2271e63b4 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:09.527838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ddc344df-60f9-40f7-ac1c-1de2271e63b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:09.570726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:09.653624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 677/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:09.954042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8697b-88cb-4119-af5a-1ba169e63ffa None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:09.955922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8af8697b-88cb-4119-af5a-1ba169e63ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:10.144621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8697b-88cb-4119-af5a-1ba169e63ffa None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:10.157380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:10.301897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38633824-5127-46e8-a046-1d104d94a323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:10.302585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633824-5127-46e8-a046-1d104d94a323 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:10.302948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633824-5127-46e8-a046-1d104d94a323 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:10.303198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633824-5127-46e8-a046-1d104d94a323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:10.304786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-38633824-5127-46e8-a046-1d104d94a323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:10.419722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38633824-5127-46e8-a046-1d104d94a323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:10.421311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38633824-5127-46e8-a046-1d104d94a323 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:10.439835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 686/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:13.280007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10594302-9d39-4a6a-9554-5178607fa280 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:13.280572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10594302-9d39-4a6a-9554-5178607fa280 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:13.280802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10594302-9d39-4a6a-9554-5178607fa280 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:13.282943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 678/1364] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:13.290033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab358-ec66-43d7-8210-7cae01d925d6 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:13.290816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab358-ec66-43d7-8210-7cae01d925d6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:13.290816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab358-ec66-43d7-8210-7cae01d925d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:13.310896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:13.372178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-b78ab358-ec66-43d7-8210-7cae01d925d6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:13.374630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b78ab358-ec66-43d7-8210-7cae01d925d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:13.494104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 687/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:13.795706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09ad8-673b-4893-8faa-26fe56c459fb None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:13.796028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09ad8-673b-4893-8faa-26fe56c459fb None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:13.796266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09ad8-673b-4893-8faa-26fe56c459fb None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:13.797936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 679/1366] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 8 13:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:13.803695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63980f2b-264c-4133-ab34-140fdacebaa9 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:13.804251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63980f2b-264c-4133-ab34-140fdacebaa9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:13.804697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63980f2b-264c-4133-ab34-140fdacebaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:13.898529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-63980f2b-264c-4133-ab34-140fdacebaa9 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:13.900021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-63980f2b-264c-4133-ab34-140fdacebaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:13.941552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:14.062975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 688/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 8 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:14.068799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f09ad8-673b-4893-8faa-26fe56c459fb None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:14.070453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e9f09ad8-673b-4893-8faa-26fe56c459fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:14.248182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f09ad8-673b-4893-8faa-26fe56c459fb None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:14.274343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:14.391605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1a74d-4fa2-4472-8b19-c97f1b449e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:14.392493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a74d-4fa2-4472-8b19-c97f1b449e20 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:14.392783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a74d-4fa2-4472-8b19-c97f1b449e20 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:14.393060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a74d-4fa2-4472-8b19-c97f1b449e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:14.395305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-15c1a74d-4fa2-4472-8b19-c97f1b449e20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:14.420518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19911bd-8903-429e-9202-f08bca07e6cd None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:14.421527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19911bd-8903-429e-9202-f08bca07e6cd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:14.421830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19911bd-8903-429e-9202-f08bca07e6cd None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:14.424277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 689/1368] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:46:14.429532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd119d7-c80b-4478-b8e2-449118d78d13 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:14.429888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd119d7-c80b-4478-b8e2-449118d78d13 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:14.430261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd119d7-c80b-4478-b8e2-449118d78d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:14.519459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1a74d-4fa2-4472-8b19-c97f1b449e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:14.521491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1a74d-4fa2-4472-8b19-c97f1b449e20 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:14.521686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1fd119d7-c80b-4478-b8e2-449118d78d13 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:14.523384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1fd119d7-c80b-4478-b8e2-449118d78d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:14.543100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 680/1369] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:14.632938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:14.641923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 690/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:46:14.989703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59049320-7385-452b-94b9-a8a07e9f9bd7 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:14.989703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59049320-7385-452b-94b9-a8a07e9f9bd7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:14.989703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59049320-7385-452b-94b9-a8a07e9f9bd7 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:14.990684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 681/1371] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 8 13:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:14.996043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfad45a-1aca-4191-93a5-afb6cb7af113 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:14.996339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfad45a-1aca-4191-93a5-afb6cb7af113 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:14.996618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfad45a-1aca-4191-93a5-afb6cb7af113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:15.085210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-6cfad45a-1aca-4191-93a5-afb6cb7af113 None None] MFA Rules not processed for user `817a190d001b40a2afee944f19270aee`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:15.086509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6cfad45a-1aca-4191-93a5-afb6cb7af113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:15.180693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 691/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 8 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:15.193572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59049320-7385-452b-94b9-a8a07e9f9bd7 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:15.196012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-59049320-7385-452b-94b9-a8a07e9f9bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:15.235264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:15.325197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59049320-7385-452b-94b9-a8a07e9f9bd7 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:15.412221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c2712-9c7b-46fa-a28c-22bbb893437d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:15.412931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2712-9c7b-46fa-a28c-22bbb893437d service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:15.413264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2712-9c7b-46fa-a28c-22bbb893437d service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:15.413549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c2712-9c7b-46fa-a28c-22bbb893437d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:15.415351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f48c2712-9c7b-46fa-a28c-22bbb893437d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:15.537473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:15.582123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c2712-9c7b-46fa-a28c-22bbb893437d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:15.583740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c2712-9c7b-46fa-a28c-22bbb893437d service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:15.616548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 682/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:17.008748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227eda-5209-4f07-94aa-8bc69725f7ff None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:17.009280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227eda-5209-4f07-94aa-8bc69725f7ff None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:17.009280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227eda-5209-4f07-94aa-8bc69725f7ff None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:17.010732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 692/1374] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:17.016999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513bc2d-0756-4604-8050-e67900194bdc None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:17.017749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513bc2d-0756-4604-8050-e67900194bdc None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:17.017951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513bc2d-0756-4604-8050-e67900194bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:17.098428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2513bc2d-0756-4604-8050-e67900194bdc None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:17.100091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2513bc2d-0756-4604-8050-e67900194bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:17.210329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 683/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:17.510345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e227eda-5209-4f07-94aa-8bc69725f7ff None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:17.511516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7e227eda-5209-4f07-94aa-8bc69725f7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:17.615508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:17.703317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e227eda-5209-4f07-94aa-8bc69725f7ff None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:17.834594 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365179e5-5ee0-4940-b00e-bba8744526d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:17.835286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365179e5-5ee0-4940-b00e-bba8744526d3 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:17.835564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365179e5-5ee0-4940-b00e-bba8744526d3 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:17.835833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365179e5-5ee0-4940-b00e-bba8744526d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:17.837507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-365179e5-5ee0-4940-b00e-bba8744526d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:17.896230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:17.949280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365179e5-5ee0-4940-b00e-bba8744526d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:17.950821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365179e5-5ee0-4940-b00e-bba8744526d3 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:17.972141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 693/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:20.393904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50704b-9b5e-436d-b116-e02f19b4a053 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:20.394927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50704b-9b5e-436d-b116-e02f19b4a053 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:20.395212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50704b-9b5e-436d-b116-e02f19b4a053 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:20.396632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 684/1377] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:20.402214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82bf90-0bea-4785-848a-43476f87bf7c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:20.402654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82bf90-0bea-4785-848a-43476f87bf7c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:20.402836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82bf90-0bea-4785-848a-43476f87bf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:20.492334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ba82bf90-0bea-4785-848a-43476f87bf7c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:20.493926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ba82bf90-0bea-4785-848a-43476f87bf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:20.615650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 694/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:20.912212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50704b-9b5e-436d-b116-e02f19b4a053 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:20.913734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0c50704b-9b5e-436d-b116-e02f19b4a053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:20.954344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:21.077687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50704b-9b5e-436d-b116-e02f19b4a053 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:21.202104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:21.212402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194510f-93ce-4d21-9e67-8f168459a90b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:21.217105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194510f-93ce-4d21-9e67-8f168459a90b service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:21.217105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194510f-93ce-4d21-9e67-8f168459a90b service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:21.217105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194510f-93ce-4d21-9e67-8f168459a90b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:21.217105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6194510f-93ce-4d21-9e67-8f168459a90b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:21.334432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194510f-93ce-4d21-9e67-8f168459a90b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:21.336065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194510f-93ce-4d21-9e67-8f168459a90b service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:21.355923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 685/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:23.788132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a796ec-5606-418f-86c7-e32523f51bc0 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:23.788682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a796ec-5606-418f-86c7-e32523f51bc0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:23.788791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a796ec-5606-418f-86c7-e32523f51bc0 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:23.790725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 695/1380] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:23.797275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b5779-2eeb-46fd-a11a-25cf68102018 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:23.797792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b5779-2eeb-46fd-a11a-25cf68102018 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:23.797925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b5779-2eeb-46fd-a11a-25cf68102018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:23.873677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e89b5779-2eeb-46fd-a11a-25cf68102018 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:23.874925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e89b5779-2eeb-46fd-a11a-25cf68102018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:23.956547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:23.998354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 686/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:24.340474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a796ec-5606-418f-86c7-e32523f51bc0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:24.342210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-26a796ec-5606-418f-86c7-e32523f51bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:24.376743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:24.517182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a796ec-5606-418f-86c7-e32523f51bc0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:24.644994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:24.671274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91669349-ac9a-47e3-ac28-23abca34d1ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:24.674266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669349-ac9a-47e3-ac28-23abca34d1ca service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:24.674266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669349-ac9a-47e3-ac28-23abca34d1ca service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:24.674266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669349-ac9a-47e3-ac28-23abca34d1ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:24.676688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-91669349-ac9a-47e3-ac28-23abca34d1ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:24.786616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91669349-ac9a-47e3-ac28-23abca34d1ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:24.788228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91669349-ac9a-47e3-ac28-23abca34d1ca service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:24.807161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 696/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:27.681527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6a41e-939b-49ea-a111-79fec77e914a None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:27.682313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6a41e-939b-49ea-a111-79fec77e914a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:27.682313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6a41e-939b-49ea-a111-79fec77e914a None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:27.683428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 687/1383] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:27.689041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60ef7-72b3-4191-98c2-54cc9900ef91 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:27.689506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60ef7-72b3-4191-98c2-54cc9900ef91 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:27.689606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60ef7-72b3-4191-98c2-54cc9900ef91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:27.774235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-06c60ef7-72b3-4191-98c2-54cc9900ef91 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:27.776005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-06c60ef7-72b3-4191-98c2-54cc9900ef91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:27.849879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:27.895640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 697/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:28.192375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6a41e-939b-49ea-a111-79fec77e914a None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:28.193927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f4c6a41e-939b-49ea-a111-79fec77e914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:28.427801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6a41e-939b-49ea-a111-79fec77e914a None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:28.493453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:28.563451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d49885-ef46-4d73-a614-ceb9ec0e378f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:28.563922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d49885-ef46-4d73-a614-ceb9ec0e378f service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:28.564204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d49885-ef46-4d73-a614-ceb9ec0e378f service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:28.564504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d49885-ef46-4d73-a614-ceb9ec0e378f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:28.566117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-73d49885-ef46-4d73-a614-ceb9ec0e378f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:28.680971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d49885-ef46-4d73-a614-ceb9ec0e378f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:28.682540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d49885-ef46-4d73-a614-ceb9ec0e378f service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:28.701210 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 688/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:31.073436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b2603-2c65-4ea0-986d-e988e92c6859 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:31.073968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b2603-2c65-4ea0-986d-e988e92c6859 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:31.073968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b2603-2c65-4ea0-986d-e988e92c6859 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:31.075447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 698/1386] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:31.081451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf413ae1-e1e1-43c7-a845-4221ee470c85 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:31.081941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf413ae1-e1e1-43c7-a845-4221ee470c85 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:31.082047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf413ae1-e1e1-43c7-a845-4221ee470c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:31.151827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-bf413ae1-e1e1-43c7-a845-4221ee470c85 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:31.152939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bf413ae1-e1e1-43c7-a845-4221ee470c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:31.161665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:31.274419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 689/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:31.557492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a60cb8-d08d-4488-8852-1706c04de061 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:31.558049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a60cb8-d08d-4488-8852-1706c04de061 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:31.558782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a60cb8-d08d-4488-8852-1706c04de061 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:31.560993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 699/1388] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:31.568172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c73df-4e12-4bca-b84e-3ee57c9b91fa None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:31.568557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c73df-4e12-4bca-b84e-3ee57c9b91fa None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:31.568838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c73df-4e12-4bca-b84e-3ee57c9b91fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:31.584788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a60cb8-d08d-4488-8852-1706c04de061 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:31.586872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-26a60cb8-d08d-4488-8852-1706c04de061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:31.646123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e76c73df-4e12-4bca-b84e-3ee57c9b91fa None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:31.648410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e76c73df-4e12-4bca-b84e-3ee57c9b91fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:31.739781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:31.751354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 690/1389] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:31.758460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a60cb8-d08d-4488-8852-1706c04de061 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:31.808685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:31.877189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e6385-8f34-4ef7-bf9c-1c66eec2fc01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:31.878608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6385-8f34-4ef7-bf9c-1c66eec2fc01 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:31.879032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6385-8f34-4ef7-bf9c-1c66eec2fc01 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:31.879426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6385-8f34-4ef7-bf9c-1c66eec2fc01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:31.881282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-845e6385-8f34-4ef7-bf9c-1c66eec2fc01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:31.978399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e6385-8f34-4ef7-bf9c-1c66eec2fc01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:31.980020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e6385-8f34-4ef7-bf9c-1c66eec2fc01 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:31.998240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 700/1390] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:46:32.054416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab4b7a-d3b4-48ea-89f8-7bd780034780 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:32.054743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab4b7a-d3b4-48ea-89f8-7bd780034780 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:32.055021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab4b7a-d3b4-48ea-89f8-7bd780034780 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:32.056537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 691/1391] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 8 13:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:32.061581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca2012-35de-4bf7-a226-12b7d13ec26b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:32.061924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca2012-35de-4bf7-a226-12b7d13ec26b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:32.062231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca2012-35de-4bf7-a226-12b7d13ec26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:32.125647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:32.144851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-2bca2012-35de-4bf7-a226-12b7d13ec26b None None] MFA Rules not processed for user `6c07c85b63514d10bb17b836c60f2824`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:32.146159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2bca2012-35de-4bf7-a226-12b7d13ec26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:32.242313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 701/1392] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 8 13:46:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:32.252822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab4b7a-d3b4-48ea-89f8-7bd780034780 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:32.254355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b5ab4b7a-d3b4-48ea-89f8-7bd780034780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:32.369418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab4b7a-d3b4-48ea-89f8-7bd780034780 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:32.445309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9e32d-5fa9-44e7-b836-4c706dd74189 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c07c85b63514d10bb17b836c60f2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:32.445926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e32d-5fa9-44e7-b836-4c706dd74189 service glance] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:32.446242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e32d-5fa9-44e7-b836-4c706dd74189 service glance] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:32.446519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e32d-5fa9-44e7-b836-4c706dd74189 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:32.448717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-43c9e32d-5fa9-44e7-b836-4c706dd74189 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:32.498805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:32.570806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9e32d-5fa9-44e7-b836-4c706dd74189 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:32.572147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9e32d-5fa9-44e7-b836-4c706dd74189 service glance] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:32.593019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 692/1393] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:33.537221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72023-502a-4cd9-a1f1-7dc0509b0b71 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:33.537712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72023-502a-4cd9-a1f1-7dc0509b0b71 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:33.537903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72023-502a-4cd9-a1f1-7dc0509b0b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:33.607759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5ac72023-502a-4cd9-a1f1-7dc0509b0b71 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:33.609520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5ac72023-502a-4cd9-a1f1-7dc0509b0b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:33.698619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 702/1394] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:33.741664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b74cf-a788-4624-b4da-ceaa5073afb4 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:33.742189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b74cf-a788-4624-b4da-ceaa5073afb4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:33.742411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b74cf-a788-4624-b4da-ceaa5073afb4 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:33.743846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 693/1395] 10.222.0.25 () {54 vars in 943 bytes} [Thu Sep 8 13:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:33.752264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc02c91-df7b-41cc-b372-8ac0150a9086 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:33.752635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc02c91-df7b-41cc-b372-8ac0150a9086 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:33.752981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc02c91-df7b-41cc-b372-8ac0150a9086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:33.763489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:33.856565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-8bc02c91-df7b-41cc-b372-8ac0150a9086 None None] MFA Rules not processed for user `136ce464d3194f98952beed2d89b8bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:33.857794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8bc02c91-df7b-41cc-b372-8ac0150a9086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:33.972064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 703/1396] 10.222.0.25 () {58 vars in 1040 bytes} [Thu Sep 8 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:33.981513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b74cf-a788-4624-b4da-ceaa5073afb4 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:33.983154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8e6b74cf-a788-4624-b4da-ceaa5073afb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:34.140445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b74cf-a788-4624-b4da-ceaa5073afb4 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:34.154642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:34.257965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:34.258603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:34.258883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:34.259155 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:34.261004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:34.329842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddda195-3463-489a-9a1c-a02f5ef40f4a None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:34.330153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddda195-3463-489a-9a1c-a02f5ef40f4a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:34.330440 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddda195-3463-489a-9a1c-a02f5ef40f4a None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:34.331868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 704/1397] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:34.336992 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2c01b-9a9b-4784-891e-5852f54d84c7 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:34.337275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2c01b-9a9b-4784-891e-5852f54d84c7 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:34.337551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2c01b-9a9b-4784-891e-5852f54d84c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:34.347742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:34.349362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:34.357653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 694/1398] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 13:46:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:46:34.409504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-49a2c01b-9a9b-4784-891e-5852f54d84c7 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:34.410688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-49a2c01b-9a9b-4784-891e-5852f54d84c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:34.427687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:34.509224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 705/1399] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:46:34.807571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:34.808788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:34.867597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:34.929337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7d47d-5ba4-4b9b-ab10-b382262f9611 service swift] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:35.085339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d8a82-2b53-4a00-9264-fc3587a7ec1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:35.086202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d8a82-2b53-4a00-9264-fc3587a7ec1c service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:35.086626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d8a82-2b53-4a00-9264-fc3587a7ec1c service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:35.087299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d8a82-2b53-4a00-9264-fc3587a7ec1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:35.089523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-999d8a82-2b53-4a00-9264-fc3587a7ec1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:35.184744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:35.217806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d8a82-2b53-4a00-9264-fc3587a7ec1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:35.220198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d8a82-2b53-4a00-9264-fc3587a7ec1c service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:35.245127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 695/1400] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:37.681866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db6e47-d75e-4668-96d7-4228d6a2c43c None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:37.681866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db6e47-d75e-4668-96d7-4228d6a2c43c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:37.681866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db6e47-d75e-4668-96d7-4228d6a2c43c None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:37.681866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 706/1401] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:37.693141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed61649-648f-4977-a70a-001e6b75bea5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:37.694273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed61649-648f-4977-a70a-001e6b75bea5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:37.694273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed61649-648f-4977-a70a-001e6b75bea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:37.782894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-aed61649-648f-4977-a70a-001e6b75bea5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:37.784753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-aed61649-648f-4977-a70a-001e6b75bea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:37.934343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 696/1402] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:38.231479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db6e47-d75e-4668-96d7-4228d6a2c43c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:38.233000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b5db6e47-d75e-4668-96d7-4228d6a2c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:38.433316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:38.435321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db6e47-d75e-4668-96d7-4228d6a2c43c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:38.607277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953786eb-1d54-4816-b2d4-19ecea883dbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:38.608032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953786eb-1d54-4816-b2d4-19ecea883dbb service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:38.608399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953786eb-1d54-4816-b2d4-19ecea883dbb service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:38.609282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953786eb-1d54-4816-b2d4-19ecea883dbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:38.611383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-953786eb-1d54-4816-b2d4-19ecea883dbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:38.740266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953786eb-1d54-4816-b2d4-19ecea883dbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:38.741916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953786eb-1d54-4816-b2d4-19ecea883dbb service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:38.763626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 707/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:41.333781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e3c3-8168-4fbc-a648-78b4eba58d00 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:41.334384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e3c3-8168-4fbc-a648-78b4eba58d00 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:41.334384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e3c3-8168-4fbc-a648-78b4eba58d00 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:41.341669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 697/1404] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:41.344161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf5de9-325a-40ed-a6f4-9b232a024176 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:41.344832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf5de9-325a-40ed-a6f4-9b232a024176 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:41.345049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf5de9-325a-40ed-a6f4-9b232a024176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:41.352548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:41.462836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d7bf5de9-325a-40ed-a6f4-9b232a024176 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:41.464477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d7bf5de9-325a-40ed-a6f4-9b232a024176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:41.624655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 708/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:41.934305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6e3c3-8168-4fbc-a648-78b4eba58d00 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:41.936035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-aef6e3c3-8168-4fbc-a648-78b4eba58d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:41.999262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:42.127547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6e3c3-8168-4fbc-a648-78b4eba58d00 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:42.283974 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b496d-0c15-494f-8dbf-42e1f33db5a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:42.284985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b496d-0c15-494f-8dbf-42e1f33db5a3 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:42.285509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b496d-0c15-494f-8dbf-42e1f33db5a3 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:42.286014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b496d-0c15-494f-8dbf-42e1f33db5a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:42.288338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-aa1b496d-0c15-494f-8dbf-42e1f33db5a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:42.291594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:42.431661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b496d-0c15-494f-8dbf-42e1f33db5a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:42.433296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b496d-0c15-494f-8dbf-42e1f33db5a3 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:42.455952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 698/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:46:45.100623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ddd57-7a67-4656-8aa4-21cec80780e6 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:45.101677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ddd57-7a67-4656-8aa4-21cec80780e6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:45.101677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ddd57-7a67-4656-8aa4-21cec80780e6 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:45.103863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 709/1407] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:46:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:46:45.116167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb561-e245-4be2-b5c2-1714e446df60 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:45.117018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb561-e245-4be2-b5c2-1714e446df60 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:45.117018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb561-e245-4be2-b5c2-1714e446df60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:45.221005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-d08cb561-e245-4be2-b5c2-1714e446df60 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:45.222799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d08cb561-e245-4be2-b5c2-1714e446df60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:45.338707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:45.381622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 699/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:45.740237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ddd57-7a67-4656-8aa4-21cec80780e6 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:45.741759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e98ddd57-7a67-4656-8aa4-21cec80780e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:45.843393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:45.935201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ddd57-7a67-4656-8aa4-21cec80780e6 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:46.117048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8be5b-66a1-4e1a-a76f-a8cc148fec92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:46.117819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8be5b-66a1-4e1a-a76f-a8cc148fec92 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:46.118413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8be5b-66a1-4e1a-a76f-a8cc148fec92 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:46.118812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8be5b-66a1-4e1a-a76f-a8cc148fec92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:46.121078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-52b8be5b-66a1-4e1a-a76f-a8cc148fec92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:46.188367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:46.247689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8be5b-66a1-4e1a-a76f-a8cc148fec92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:46.249659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8be5b-66a1-4e1a-a76f-a8cc148fec92 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:46.273580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 710/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:55.377947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=9 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:56.270732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=25 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:59.216755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c10b9-bed8-4760-8283-24ad50d79315 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:59.217287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c10b9-bed8-4760-8283-24ad50d79315 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:59.217287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c10b9-bed8-4760-8283-24ad50d79315 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:59.219611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 700/1410] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 8 13:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:59.225204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacae77d-35bf-41ce-9a87-0b5c83d841a9 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:59.225678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacae77d-35bf-41ce-9a87-0b5c83d841a9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:59.225823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacae77d-35bf-41ce-9a87-0b5c83d841a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:59.311721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-eacae77d-35bf-41ce-9a87-0b5c83d841a9 None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:59.312972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-eacae77d-35bf-41ce-9a87-0b5c83d841a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:59.422651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 711/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 8 13:46:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:59.437238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954a24e-3cbf-40f3-81e7-017673cf3ac2 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:59.437530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954a24e-3cbf-40f3-81e7-017673cf3ac2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:59.437809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954a24e-3cbf-40f3-81e7-017673cf3ac2 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:59.439256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 701/1412] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 8 13:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:59.444126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e48fe-1ee3-453a-a61e-c27945e304b5 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:59.444420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e48fe-1ee3-453a-a61e-c27945e304b5 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:59.444694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e48fe-1ee3-453a-a61e-c27945e304b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:59.518545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:59.521553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-431e48fe-1ee3-453a-a61e-c27945e304b5 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:59.522875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-431e48fe-1ee3-453a-a61e-c27945e304b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:59.649220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 712/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 8 13:46:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:02.298019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f27ba-ea2e-49f0-be31-f7bf76951802 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:02.298632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f27ba-ea2e-49f0-be31-f7bf76951802 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:02.298632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f27ba-ea2e-49f0-be31-f7bf76951802 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:02.299940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:02.305361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63671-acc7-4579-851f-b831a0f2d4f9 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:02.305797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63671-acc7-4579-851f-b831a0f2d4f9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:02.305919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63671-acc7-4579-851f-b831a0f2d4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:02.375775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-98c63671-acc7-4579-851f-b831a0f2d4f9 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:02.377015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-98c63671-acc7-4579-851f-b831a0f2d4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:02.432345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:02.475747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:02.767950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f27ba-ea2e-49f0-be31-f7bf76951802 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:02.769479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-251f27ba-ea2e-49f0-be31-f7bf76951802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:02.921920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f27ba-ea2e-49f0-be31-f7bf76951802 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:02.995925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:03.038986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a538f16-d882-42d0-b561-202f2761173a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:03.039593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538f16-d882-42d0-b561-202f2761173a service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:03.039962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538f16-d882-42d0-b561-202f2761173a service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:03.040235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538f16-d882-42d0-b561-202f2761173a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:03.041866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0a538f16-d882-42d0-b561-202f2761173a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:03.139434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a538f16-d882-42d0-b561-202f2761173a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:03.141074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a538f16-d882-42d0-b561-202f2761173a service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:03.161165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 703/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:04.318708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5131f03-76a1-46b7-ab32-4223420d6a7b None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:04.319256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5131f03-76a1-46b7-ab32-4223420d6a7b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:04.319256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5131f03-76a1-46b7-ab32-4223420d6a7b None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:04.320602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:04.327161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c31424-e14a-4048-b397-f1bab542dea4 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:04.345928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c31424-e14a-4048-b397-f1bab542dea4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:04.345928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c31424-e14a-4048-b397-f1bab542dea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:04.408046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-14c31424-e14a-4048-b397-f1bab542dea4 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:04.409284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-14c31424-e14a-4048-b397-f1bab542dea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:04.436391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:04.508160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:04.659233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5131f03-76a1-46b7-ab32-4223420d6a7b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:04.660773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d5131f03-76a1-46b7-ab32-4223420d6a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:04.805707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5131f03-76a1-46b7-ab32-4223420d6a7b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:04.846976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:04.922275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda14fb-9c06-4262-bc3b-133a969653e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:04.922895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda14fb-9c06-4262-bc3b-133a969653e4 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:04.923174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda14fb-9c06-4262-bc3b-133a969653e4 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:04.923450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda14fb-9c06-4262-bc3b-133a969653e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:04.925050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6cda14fb-9c06-4262-bc3b-133a969653e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:05.015993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda14fb-9c06-4262-bc3b-133a969653e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:05.017653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda14fb-9c06-4262-bc3b-133a969653e4 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:05.037264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 715/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:06.192759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00673df-53d6-476e-a2e8-1a4704f40207 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:06.193331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00673df-53d6-476e-a2e8-1a4704f40207 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:06.193331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00673df-53d6-476e-a2e8-1a4704f40207 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:06.194706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:06.201058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6c0ec-fcda-4b34-842f-abf0d41bdc04 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:06.201489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6c0ec-fcda-4b34-842f-abf0d41bdc04 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:06.201608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6c0ec-fcda-4b34-842f-abf0d41bdc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:06.258744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:06.281672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5ab6c0ec-fcda-4b34-842f-abf0d41bdc04 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:06.282989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5ab6c0ec-fcda-4b34-842f-abf0d41bdc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:06.382649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:06.534578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00673df-53d6-476e-a2e8-1a4704f40207 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:06.536328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c00673df-53d6-476e-a2e8-1a4704f40207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:06.684526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00673df-53d6-476e-a2e8-1a4704f40207 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:06.697214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:06.799444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89510706-d2b3-4d52-845d-4dd5ee502346 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:06.800257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89510706-d2b3-4d52-845d-4dd5ee502346 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:06.800546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89510706-d2b3-4d52-845d-4dd5ee502346 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:06.800832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89510706-d2b3-4d52-845d-4dd5ee502346 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:06.802439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-89510706-d2b3-4d52-845d-4dd5ee502346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:06.893726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89510706-d2b3-4d52-845d-4dd5ee502346 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:06.895361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89510706-d2b3-4d52-845d-4dd5ee502346 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:06.913775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 706/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:08.030828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809571a-c27d-4b00-88f6-8879a43851f0 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:08.031393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809571a-c27d-4b00-88f6-8879a43851f0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:08.031393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809571a-c27d-4b00-88f6-8879a43851f0 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:08.032719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:08.039384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42deba-ecd0-4676-9496-cce4972ef1dd None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:08.039829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42deba-ecd0-4676-9496-cce4972ef1dd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:08.040029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42deba-ecd0-4676-9496-cce4972ef1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:08.061094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:08.234553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-fa42deba-ecd0-4676-9496-cce4972ef1dd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:08.235777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fa42deba-ecd0-4676-9496-cce4972ef1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:08.333022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:08.484838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809571a-c27d-4b00-88f6-8879a43851f0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:08.486452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f809571a-c27d-4b00-88f6-8879a43851f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:08.599062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:08.635418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809571a-c27d-4b00-88f6-8879a43851f0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:08.749330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f0de5-dda2-4164-ab79-825d989a055c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:08.749931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0de5-dda2-4164-ab79-825d989a055c service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:08.750237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0de5-dda2-4164-ab79-825d989a055c service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:08.750508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0de5-dda2-4164-ab79-825d989a055c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:08.752127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f22f0de5-dda2-4164-ab79-825d989a055c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:08.830416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:08.844217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f0de5-dda2-4164-ab79-825d989a055c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:08.845809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f0de5-dda2-4164-ab79-825d989a055c service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:08.864675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 718/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:10.007744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6328-b613-4b50-8bff-78ea9baac962 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:10.008321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6328-b613-4b50-8bff-78ea9baac962 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:10.008321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6328-b613-4b50-8bff-78ea9baac962 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:10.009634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:10.017125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a229d6d-730b-4420-b7d0-aeff778437f6 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:10.017640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a229d6d-730b-4420-b7d0-aeff778437f6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:10.017744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a229d6d-730b-4420-b7d0-aeff778437f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:10.092357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-7a229d6d-730b-4420-b7d0-aeff778437f6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:10.093619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7a229d6d-730b-4420-b7d0-aeff778437f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:10.192717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:10.343163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf6328-b613-4b50-8bff-78ea9baac962 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:10.344685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-80bf6328-b613-4b50-8bff-78ea9baac962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:10.433529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:10.496585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf6328-b613-4b50-8bff-78ea9baac962 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:10.610886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd85c76-0c56-4857-b1c5-5762d72c0f1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:10.611414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd85c76-0c56-4857-b1c5-5762d72c0f1e service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:10.611687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd85c76-0c56-4857-b1c5-5762d72c0f1e service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:10.611954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd85c76-0c56-4857-b1c5-5762d72c0f1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:10.613700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ddd85c76-0c56-4857-b1c5-5762d72c0f1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:10.663542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:10.703095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd85c76-0c56-4857-b1c5-5762d72c0f1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:10.707945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd85c76-0c56-4857-b1c5-5762d72c0f1e service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:10.729758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 709/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:11.919426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df3eb-6294-4472-beb3-4c9a33295755 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:11.920010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df3eb-6294-4472-beb3-4c9a33295755 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:11.920010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df3eb-6294-4472-beb3-4c9a33295755 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:11.921314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:11.928677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb316-e74f-4a58-bd27-2d06ea89b188 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:11.929261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb316-e74f-4a58-bd27-2d06ea89b188 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:11.929377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb316-e74f-4a58-bd27-2d06ea89b188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:12.007227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-1c0bb316-e74f-4a58-bd27-2d06ea89b188 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:12.008458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1c0bb316-e74f-4a58-bd27-2d06ea89b188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:12.110446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:12.261317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402df3eb-6294-4472-beb3-4c9a33295755 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:12.263287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-402df3eb-6294-4472-beb3-4c9a33295755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:12.313209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:12.412908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402df3eb-6294-4472-beb3-4c9a33295755 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:12.534198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f393-a4c2-4927-9f89-fd3fa33671a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:12.534728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f393-a4c2-4927-9f89-fd3fa33671a9 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:12.535042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f393-a4c2-4927-9f89-fd3fa33671a9 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:12.535314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f393-a4c2-4927-9f89-fd3fa33671a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:12.536952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c4b6f393-a4c2-4927-9f89-fd3fa33671a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:12.544376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:12.628721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6f393-a4c2-4927-9f89-fd3fa33671a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:12.630371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6f393-a4c2-4927-9f89-fd3fa33671a9 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:12.650213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 721/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:13.806212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170d881-ea5b-403b-8bc9-08cd8b5d6a47 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:13.806795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170d881-ea5b-403b-8bc9-08cd8b5d6a47 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:13.806902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170d881-ea5b-403b-8bc9-08cd8b5d6a47 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:13.809521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:47:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:13.816857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cf540-c8de-4123-98ac-19b7f7dcb7bd None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:13.817385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cf540-c8de-4123-98ac-19b7f7dcb7bd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:13.817385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cf540-c8de-4123-98ac-19b7f7dcb7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:13.907713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-429cf540-c8de-4123-98ac-19b7f7dcb7bd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:13.908961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-429cf540-c8de-4123-98ac-19b7f7dcb7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:13.990078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:14.023638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:14.176379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170d881-ea5b-403b-8bc9-08cd8b5d6a47 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:14.177811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d170d881-ea5b-403b-8bc9-08cd8b5d6a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:14.195779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:14.325570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170d881-ea5b-403b-8bc9-08cd8b5d6a47 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:14.415086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:14.441096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19f82d-791f-4bda-9fde-324047441098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:14.441633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19f82d-791f-4bda-9fde-324047441098 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:14.441912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19f82d-791f-4bda-9fde-324047441098 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:14.442212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19f82d-791f-4bda-9fde-324047441098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:14.443915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ba19f82d-791f-4bda-9fde-324047441098 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:14.541840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19f82d-791f-4bda-9fde-324047441098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:14.543613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19f82d-791f-4bda-9fde-324047441098 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:14.561789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 712/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:15.733123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9784d-8616-4783-a34a-53ad429351aa None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:15.733689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9784d-8616-4783-a34a-53ad429351aa None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:15.733689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9784d-8616-4783-a34a-53ad429351aa None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:15.735750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:15.742456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdac5a3-a1e4-4f31-9ff4-ebfc1013e935 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:15.742916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdac5a3-a1e4-4f31-9ff4-ebfc1013e935 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:15.743002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdac5a3-a1e4-4f31-9ff4-ebfc1013e935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:15.818875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2fdac5a3-a1e4-4f31-9ff4-ebfc1013e935 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:15.820092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2fdac5a3-a1e4-4f31-9ff4-ebfc1013e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:15.861657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:15.918130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:16.080010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9784d-8616-4783-a34a-53ad429351aa None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:16.081517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-66a9784d-8616-4783-a34a-53ad429351aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:16.232827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9784d-8616-4783-a34a-53ad429351aa None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:16.289419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:16.350559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef7bc3-122d-440e-af92-999fd58a6832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:16.351173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7bc3-122d-440e-af92-999fd58a6832 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:16.351453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7bc3-122d-440e-af92-999fd58a6832 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:16.351751 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7bc3-122d-440e-af92-999fd58a6832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:16.353407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-47ef7bc3-122d-440e-af92-999fd58a6832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:16.449129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef7bc3-122d-440e-af92-999fd58a6832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:16.450728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef7bc3-122d-440e-af92-999fd58a6832 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:16.468842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 724/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:17.666639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d8d60-dcd7-4dd6-a974-7bce61ab531f None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:17.667205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d8d60-dcd7-4dd6-a974-7bce61ab531f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:17.667205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d8d60-dcd7-4dd6-a974-7bce61ab531f None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:17.668584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:17.675186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97302aff-2b36-4727-9538-49aad8929786 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:17.681062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97302aff-2b36-4727-9538-49aad8929786 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:17.681062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97302aff-2b36-4727-9538-49aad8929786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:17.756972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-97302aff-2b36-4727-9538-49aad8929786 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:17.758251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-97302aff-2b36-4727-9538-49aad8929786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:17.760400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:17.855761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:18.016482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d8d60-dcd7-4dd6-a974-7bce61ab531f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:18.018649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ff6d8d60-dcd7-4dd6-a974-7bce61ab531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:18.166178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d8d60-dcd7-4dd6-a974-7bce61ab531f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:18.193591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:18.281971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43472e-11d6-42d2-b085-6c698204aac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:18.282612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43472e-11d6-42d2-b085-6c698204aac4 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:18.282893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43472e-11d6-42d2-b085-6c698204aac4 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:18.283172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43472e-11d6-42d2-b085-6c698204aac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:18.284824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-af43472e-11d6-42d2-b085-6c698204aac4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:18.375969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43472e-11d6-42d2-b085-6c698204aac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:18.378208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43472e-11d6-42d2-b085-6c698204aac4 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:18.395916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 715/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:20.481503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289d95b-592c-4ea3-827e-c658bd6f0de5 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:20.482231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289d95b-592c-4ea3-827e-c658bd6f0de5 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:20.482231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289d95b-592c-4ea3-827e-c658bd6f0de5 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:20.483469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:20.488874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3975e5-87f4-4bb5-8f5e-b8bcb43e59a5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:20.489274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3975e5-87f4-4bb5-8f5e-b8bcb43e59a5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:20.489420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3975e5-87f4-4bb5-8f5e-b8bcb43e59a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:20.531366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:20.557795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-da3975e5-87f4-4bb5-8f5e-b8bcb43e59a5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:20.559057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-da3975e5-87f4-4bb5-8f5e-b8bcb43e59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:20.658474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:20.951713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289d95b-592c-4ea3-827e-c658bd6f0de5 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:20.953336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5289d95b-592c-4ea3-827e-c658bd6f0de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:21.060092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289d95b-592c-4ea3-827e-c658bd6f0de5 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:21.091895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:21.134830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713cca8-0e0e-4b95-b8fd-fd34cdbf58ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:21.135437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713cca8-0e0e-4b95-b8fd-fd34cdbf58ab service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:21.135717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713cca8-0e0e-4b95-b8fd-fd34cdbf58ab service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:21.135995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713cca8-0e0e-4b95-b8fd-fd34cdbf58ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:21.137596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b713cca8-0e0e-4b95-b8fd-fd34cdbf58ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:21.248797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713cca8-0e0e-4b95-b8fd-fd34cdbf58ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:21.250170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713cca8-0e0e-4b95-b8fd-fd34cdbf58ab service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:21.268721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 727/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:23.609770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f4c2-0fa6-49f3-aad8-a49091ca76cb None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:23.610895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f4c2-0fa6-49f3-aad8-a49091ca76cb None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:23.610895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f4c2-0fa6-49f3-aad8-a49091ca76cb None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:23.612233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:23.617793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff6949-c643-4806-8cef-d9c541225422 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:23.618245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff6949-c643-4806-8cef-d9c541225422 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:23.618395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff6949-c643-4806-8cef-d9c541225422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:23.774794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-97ff6949-c643-4806-8cef-d9c541225422 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:23.774794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-97ff6949-c643-4806-8cef-d9c541225422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:23.774794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:23.793109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:24.078187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9f4c2-0fa6-49f3-aad8-a49091ca76cb None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:24.080393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4ed9f4c2-0fa6-49f3-aad8-a49091ca76cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:24.184964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9f4c2-0fa6-49f3-aad8-a49091ca76cb None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:24.198603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:24.259770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff4c53-4688-4d0b-9985-991688a2fdaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:24.260394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff4c53-4688-4d0b-9985-991688a2fdaa service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:24.260686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff4c53-4688-4d0b-9985-991688a2fdaa service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:24.261080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff4c53-4688-4d0b-9985-991688a2fdaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:24.262813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-eeff4c53-4688-4d0b-9985-991688a2fdaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:24.374633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff4c53-4688-4d0b-9985-991688a2fdaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:24.376817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff4c53-4688-4d0b-9985-991688a2fdaa service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:24.394824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 718/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:27.018282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1b69-ea45-4391-b8c6-5786adccefca None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:27.018850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1b69-ea45-4391-b8c6-5786adccefca None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:27.018850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1b69-ea45-4391-b8c6-5786adccefca None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:27.020185 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:27.025675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002d09-1a60-48e8-86d0-6d9c4fdf684e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:27.026107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002d09-1a60-48e8-86d0-6d9c4fdf684e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:27.026244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002d09-1a60-48e8-86d0-6d9c4fdf684e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:27.074661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:27.092895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-26002d09-1a60-48e8-86d0-6d9c4fdf684e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:27.094121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-26002d09-1a60-48e8-86d0-6d9c4fdf684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:27.197518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:27.494755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b1b69-ea45-4391-b8c6-5786adccefca None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:27.496304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f13b1b69-ea45-4391-b8c6-5786adccefca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:27.602711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b1b69-ea45-4391-b8c6-5786adccefca None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:27.645937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:27.679797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca3869-6215-4d47-9e54-c9dfc2a110c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:27.680468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3869-6215-4d47-9e54-c9dfc2a110c4 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:27.680744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3869-6215-4d47-9e54-c9dfc2a110c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:27.681021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3869-6215-4d47-9e54-c9dfc2a110c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:27.682644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4aca3869-6215-4d47-9e54-c9dfc2a110c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:27.796581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca3869-6215-4d47-9e54-c9dfc2a110c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:27.798590 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca3869-6215-4d47-9e54-c9dfc2a110c4 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:27.817604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 730/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:30.172175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10736f9-b493-41dc-8514-5e97b2c653c2 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:30.172778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10736f9-b493-41dc-8514-5e97b2c653c2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:30.173417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10736f9-b493-41dc-8514-5e97b2c653c2 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:30.174936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:30.180319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a8e18-16b5-43b1-9a83-cef33ca356da None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:30.180795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a8e18-16b5-43b1-9a83-cef33ca356da None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:30.180896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a8e18-16b5-43b1-9a83-cef33ca356da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:30.248711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-699a8e18-16b5-43b1-9a83-cef33ca356da None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:30.249968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-699a8e18-16b5-43b1-9a83-cef33ca356da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:30.271370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:30.350411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:30.639596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10736f9-b493-41dc-8514-5e97b2c653c2 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:30.640924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c10736f9-b493-41dc-8514-5e97b2c653c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:30.745155 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10736f9-b493-41dc-8514-5e97b2c653c2 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:30.766643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:30.826967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39662065-8b17-4642-8001-9f1408558867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:30.827495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39662065-8b17-4642-8001-9f1408558867 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:30.827775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39662065-8b17-4642-8001-9f1408558867 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:30.828049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39662065-8b17-4642-8001-9f1408558867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:30.830389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-39662065-8b17-4642-8001-9f1408558867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:30.940555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39662065-8b17-4642-8001-9f1408558867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:30.941990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39662065-8b17-4642-8001-9f1408558867 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:30.960554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 721/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:33.242770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dfd2b-d52c-4e23-9525-53246948fa76 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:33.243330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dfd2b-d52c-4e23-9525-53246948fa76 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:33.243330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dfd2b-d52c-4e23-9525-53246948fa76 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:33.244849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:33.251388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29177f8-038e-4aed-815f-692e4578eb74 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:33.251849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29177f8-038e-4aed-815f-692e4578eb74 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:33.251954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29177f8-038e-4aed-815f-692e4578eb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:33.324005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:33.326962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e29177f8-038e-4aed-815f-692e4578eb74 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:33.328219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e29177f8-038e-4aed-815f-692e4578eb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:33.429417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:47:33.719779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dfd2b-d52c-4e23-9525-53246948fa76 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:33.721420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a56dfd2b-d52c-4e23-9525-53246948fa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:33.827742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dfd2b-d52c-4e23-9525-53246948fa76 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:33.878496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:33.905336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb182056-2ba9-4e90-b146-a1748e063ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:33.905336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb182056-2ba9-4e90-b146-a1748e063ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:33.905336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb182056-2ba9-4e90-b146-a1748e063ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:33.905336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb182056-2ba9-4e90-b146-a1748e063ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:33.906018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-eb182056-2ba9-4e90-b146-a1748e063ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:34.020062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb182056-2ba9-4e90-b146-a1748e063ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:34.021596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb182056-2ba9-4e90-b146-a1748e063ae4 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:34.040554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 733/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:36.349701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a4db5-a5f8-40c8-bbce-77d5695b74c6 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:36.350323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a4db5-a5f8-40c8-bbce-77d5695b74c6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:36.350323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a4db5-a5f8-40c8-bbce-77d5695b74c6 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:36.351633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:36.357480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06bb6-88ad-4c01-b8d3-f5e69514435b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:36.357989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06bb6-88ad-4c01-b8d3-f5e69514435b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:36.358088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06bb6-88ad-4c01-b8d3-f5e69514435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:36.427752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f4b06bb6-88ad-4c01-b8d3-f5e69514435b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:36.428970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f4b06bb6-88ad-4c01-b8d3-f5e69514435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:36.459004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:36.530721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:36.819729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a4db5-a5f8-40c8-bbce-77d5695b74c6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:36.821259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-676a4db5-a5f8-40c8-bbce-77d5695b74c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:36.933991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a4db5-a5f8-40c8-bbce-77d5695b74c6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:36.964906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:37.018808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1db1a1-1ade-43e3-9a5b-24de407f55b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:37.019410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1db1a1-1ade-43e3-9a5b-24de407f55b9 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:37.019691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1db1a1-1ade-43e3-9a5b-24de407f55b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:37.019962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1db1a1-1ade-43e3-9a5b-24de407f55b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:37.021593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5b1db1a1-1ade-43e3-9a5b-24de407f55b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:37.141757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1db1a1-1ade-43e3-9a5b-24de407f55b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:37.143240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1db1a1-1ade-43e3-9a5b-24de407f55b9 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:37.162523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 724/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:47:39.429767 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9955f-4c44-4ed9-928c-10fd71d7315f None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:39.430381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9955f-4c44-4ed9-928c-10fd71d7315f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:39.430381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9955f-4c44-4ed9-928c-10fd71d7315f None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:39.431696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:39.437304 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a84d0e-92d8-43c7-b569-58e38396534f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:39.437760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a84d0e-92d8-43c7-b569-58e38396534f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:39.437870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a84d0e-92d8-43c7-b569-58e38396534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:39.502981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-a8a84d0e-92d8-43c7-b569-58e38396534f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:39.504205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a8a84d0e-92d8-43c7-b569-58e38396534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:39.512460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:39.601323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:39.890321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9955f-4c44-4ed9-928c-10fd71d7315f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:39.892488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-30c9955f-4c44-4ed9-928c-10fd71d7315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:39.995842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9955f-4c44-4ed9-928c-10fd71d7315f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:40.052651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:40.072379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5f9be-647d-460d-bb0e-0e8917fddf44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:40.072988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f9be-647d-460d-bb0e-0e8917fddf44 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:40.073268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f9be-647d-460d-bb0e-0e8917fddf44 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:40.073540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5f9be-647d-460d-bb0e-0e8917fddf44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:40.075202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bdf5f9be-647d-460d-bb0e-0e8917fddf44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:40.192390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5f9be-647d-460d-bb0e-0e8917fddf44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:40.194094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5f9be-647d-460d-bb0e-0e8917fddf44 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:40.215133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 736/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:42.507256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb2fa-e5b2-4756-b3f7-974cc87608ce None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:42.507868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb2fa-e5b2-4756-b3f7-974cc87608ce None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:42.507868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb2fa-e5b2-4756-b3f7-974cc87608ce None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:42.509241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:42.514525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef443292-48a7-45d7-8d0a-4ff35a0cec5c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:42.514972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef443292-48a7-45d7-8d0a-4ff35a0cec5c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:42.515084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef443292-48a7-45d7-8d0a-4ff35a0cec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:42.581447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ef443292-48a7-45d7-8d0a-4ff35a0cec5c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:42.582700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ef443292-48a7-45d7-8d0a-4ff35a0cec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:42.621862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:42.681504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:47:42.967854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cb2fa-e5b2-4756-b3f7-974cc87608ce None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:42.969929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9e1cb2fa-e5b2-4756-b3f7-974cc87608ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:43.076482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cb2fa-e5b2-4756-b3f7-974cc87608ce None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:43.115712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:43.151194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d8d20-56f1-4eac-b4a6-ab4f88636b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:43.151808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d8d20-56f1-4eac-b4a6-ab4f88636b26 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:43.152087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d8d20-56f1-4eac-b4a6-ab4f88636b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:43.152362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d8d20-56f1-4eac-b4a6-ab4f88636b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:43.153975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-285d8d20-56f1-4eac-b4a6-ab4f88636b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:43.263379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d8d20-56f1-4eac-b4a6-ab4f88636b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:43.264734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d8d20-56f1-4eac-b4a6-ab4f88636b26 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:43.284174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 727/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:45.500421 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480627ff-2148-4db6-81de-0e03eb4cc81b None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:45.501087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480627ff-2148-4db6-81de-0e03eb4cc81b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:45.501087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480627ff-2148-4db6-81de-0e03eb4cc81b None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:45.502392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:45.507643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9a4f8-7781-4a4a-9b30-f5d1632cc9da None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:45.508139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9a4f8-7781-4a4a-9b30-f5d1632cc9da None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:45.508262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9a4f8-7781-4a4a-9b30-f5d1632cc9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:45.573089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-abe9a4f8-7781-4a4a-9b30-f5d1632cc9da None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:45.574334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-abe9a4f8-7781-4a4a-9b30-f5d1632cc9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:45.592830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:45.672778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:45.954704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480627ff-2148-4db6-81de-0e03eb4cc81b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:45.956276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-480627ff-2148-4db6-81de-0e03eb4cc81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:46.064247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480627ff-2148-4db6-81de-0e03eb4cc81b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:46.133220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:46.139937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955421db-03b7-4ba6-9ce0-2d2433367769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:46.147215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955421db-03b7-4ba6-9ce0-2d2433367769 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:46.147215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955421db-03b7-4ba6-9ce0-2d2433367769 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:46.147215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955421db-03b7-4ba6-9ce0-2d2433367769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:46.147215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-955421db-03b7-4ba6-9ce0-2d2433367769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:46.255630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955421db-03b7-4ba6-9ce0-2d2433367769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:46.257002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955421db-03b7-4ba6-9ce0-2d2433367769 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:46.275697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 739/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:48.508664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0b5ec-fc49-4d86-96c7-fb5ca0f82cc1 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:48.509191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0b5ec-fc49-4d86-96c7-fb5ca0f82cc1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:48.509276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0b5ec-fc49-4d86-96c7-fb5ca0f82cc1 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:48.511083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:47:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:48.516956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea66465-26aa-425e-857a-57dacccfaf55 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:48.517580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea66465-26aa-425e-857a-57dacccfaf55 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:48.517580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea66465-26aa-425e-857a-57dacccfaf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:48.582678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0ea66465-26aa-425e-857a-57dacccfaf55 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:48.583893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0ea66465-26aa-425e-857a-57dacccfaf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:48.632004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:48.682061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:48.995976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0b5ec-fc49-4d86-96c7-fb5ca0f82cc1 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:48.997520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cdf0b5ec-fc49-4d86-96c7-fb5ca0f82cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:49.100138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0b5ec-fc49-4d86-96c7-fb5ca0f82cc1 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:49.147697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:49.184622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1b4cf-fc95-4d5c-8a80-f7e89a157a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:49.190298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1b4cf-fc95-4d5c-8a80-f7e89a157a87 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:49.190576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1b4cf-fc95-4d5c-8a80-f7e89a157a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:49.190847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1b4cf-fc95-4d5c-8a80-f7e89a157a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:49.192504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-59c1b4cf-fc95-4d5c-8a80-f7e89a157a87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:49.318657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1b4cf-fc95-4d5c-8a80-f7e89a157a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:49.320102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1b4cf-fc95-4d5c-8a80-f7e89a157a87 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:49.338535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 730/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:58.678885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=22 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:59.336092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=66 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:09.526525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ea3d-0f9c-4a7d-b404-3569afda1029 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:09.527092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ea3d-0f9c-4a7d-b404-3569afda1029 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:09.527092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ea3d-0f9c-4a7d-b404-3569afda1029 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:09.528447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 741/1471] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:09.534003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95e6b4-dba7-4726-9f82-ba0a56adfa5a None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:09.534479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95e6b4-dba7-4726-9f82-ba0a56adfa5a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:09.534587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95e6b4-dba7-4726-9f82-ba0a56adfa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:09.601870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9a95e6b4-dba7-4726-9f82-ba0a56adfa5a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:09.603202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9a95e6b4-dba7-4726-9f82-ba0a56adfa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:09.704632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 731/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:10.005131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971fe9-6e2e-4789-a692-18d859b61f7a None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:10.005417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971fe9-6e2e-4789-a692-18d859b61f7a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:10.005698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971fe9-6e2e-4789-a692-18d859b61f7a None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:10.007144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 742/1473] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 8 13:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:10.012162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ddaee-cfca-4078-97e4-6b1eebe6d6b1 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:10.012451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ddaee-cfca-4078-97e4-6b1eebe6d6b1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:10.012729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ddaee-cfca-4078-97e4-6b1eebe6d6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:10.088939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f98ddaee-cfca-4078-97e4-6b1eebe6d6b1 None None] MFA Rules not processed for user `6c07c85b63514d10bb17b836c60f2824`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:10.090186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f98ddaee-cfca-4078-97e4-6b1eebe6d6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:10.111141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:10.166841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 732/1474] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 8 13:48:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:10.174632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971fe9-6e2e-4789-a692-18d859b61f7a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:10.176142 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b6971fe9-6e2e-4789-a692-18d859b61f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:10.279548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971fe9-6e2e-4789-a692-18d859b61f7a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:10.353375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee210b-26e8-4e60-802b-710e47b7da8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c07c85b63514d10bb17b836c60f2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:10.353989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee210b-26e8-4e60-802b-710e47b7da8e service glance] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:10.354306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee210b-26e8-4e60-802b-710e47b7da8e service glance] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:10.354579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee210b-26e8-4e60-802b-710e47b7da8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:10.356212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-48ee210b-26e8-4e60-802b-710e47b7da8e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:10.412577 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:10.470072 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee210b-26e8-4e60-802b-710e47b7da8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:10.471604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee210b-26e8-4e60-802b-710e47b7da8e service glance] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:10.490330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 743/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:48:12.732840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd140f-617a-47ed-954d-98e40644454d None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:12.733436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd140f-617a-47ed-954d-98e40644454d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:12.733436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd140f-617a-47ed-954d-98e40644454d None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:12.734832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 733/1476] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:12.740496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e664b5-671b-40ea-9d16-9a78fbf6f2ad None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:12.741035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e664b5-671b-40ea-9d16-9a78fbf6f2ad None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:12.741035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e664b5-671b-40ea-9d16-9a78fbf6f2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:12.807857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-b0e664b5-671b-40ea-9d16-9a78fbf6f2ad None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:12.809119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b0e664b5-671b-40ea-9d16-9a78fbf6f2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:12.913846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 744/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:13.192233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd140f-617a-47ed-954d-98e40644454d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:13.193694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-96bd140f-617a-47ed-954d-98e40644454d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:13.325386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd140f-617a-47ed-954d-98e40644454d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:13.379208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:13.430979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d5e4d-a549-46ce-9b62-25744f99d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:13.431578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5e4d-a549-46ce-9b62-25744f99d579 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:13.431860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5e4d-a549-46ce-9b62-25744f99d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:13.432132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5e4d-a549-46ce-9b62-25744f99d579 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:13.432779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d5e4d-a549-46ce-9b62-25744f99d579 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:13.434560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING py.warnings [None req-2e0d5e4d-a549-46ce-9b62-25744f99d579 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:13.434560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: warnings.warn(msg) Sep 08 13:48:13.434560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: Sep 08 13:48:13.435706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d5e4d-a549-46ce-9b62-25744f99d579 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:13.456466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 734/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 13:48:13 2022] GET /identity/v3/projects => generated 2553 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:48:15.747655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369853-d1db-4aa6-b999-f36c31551b58 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:15.748251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369853-d1db-4aa6-b999-f36c31551b58 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:15.748251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369853-d1db-4aa6-b999-f36c31551b58 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:15.749574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 745/1479] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:15.755379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60e57d-5dad-4aec-bb8c-59494e33babf None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:15.755915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60e57d-5dad-4aec-bb8c-59494e33babf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:15.755915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60e57d-5dad-4aec-bb8c-59494e33babf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:15.829820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-fb60e57d-5dad-4aec-bb8c-59494e33babf None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:15.831118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fb60e57d-5dad-4aec-bb8c-59494e33babf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:15.944963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 735/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:16.238389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369853-d1db-4aa6-b999-f36c31551b58 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:16.239951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-07369853-d1db-4aa6-b999-f36c31551b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:16.250367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:16.401063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369853-d1db-4aa6-b999-f36c31551b58 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:16.491863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:16.534439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59080dbe-0c55-4f2b-a2b9-92ebe5eb58f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:16.535059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080dbe-0c55-4f2b-a2b9-92ebe5eb58f9 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:16.535343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080dbe-0c55-4f2b-a2b9-92ebe5eb58f9 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:16.535615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080dbe-0c55-4f2b-a2b9-92ebe5eb58f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:16.537289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-59080dbe-0c55-4f2b-a2b9-92ebe5eb58f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:16.763711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59080dbe-0c55-4f2b-a2b9-92ebe5eb58f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:16.765236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59080dbe-0c55-4f2b-a2b9-92ebe5eb58f9 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:16.787347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 746/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:48:19.293156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108a9dd-d963-4360-8393-bc70c0063a43 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:19.293692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108a9dd-d963-4360-8393-bc70c0063a43 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:19.293692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108a9dd-d963-4360-8393-bc70c0063a43 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:19.295032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 736/1482] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:19.300537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c782b5-bffb-492b-8046-797fa5d75e2b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:19.310605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c782b5-bffb-492b-8046-797fa5d75e2b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:19.310605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c782b5-bffb-492b-8046-797fa5d75e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:19.373601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d1c782b5-bffb-492b-8046-797fa5d75e2b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:19.374869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d1c782b5-bffb-492b-8046-797fa5d75e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:19.423209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:19.478551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 747/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:19.792776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108a9dd-d963-4360-8393-bc70c0063a43 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:19.794460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4108a9dd-d963-4360-8393-bc70c0063a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:19.821510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:19.910900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108a9dd-d963-4360-8393-bc70c0063a43 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:19.989855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da289b-b2b8-4234-b3ec-8d0a63f623ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:19.990504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da289b-b2b8-4234-b3ec-8d0a63f623ad service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:19.990784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da289b-b2b8-4234-b3ec-8d0a63f623ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:19.991056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da289b-b2b8-4234-b3ec-8d0a63f623ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:19.992931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a0da289b-b2b8-4234-b3ec-8d0a63f623ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:20.071522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:20.109784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da289b-b2b8-4234-b3ec-8d0a63f623ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:20.111387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da289b-b2b8-4234-b3ec-8d0a63f623ad service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:20.130227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 737/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:48:22.388150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da107-92d3-4ca8-b8b7-44d49a383e2a None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:22.388743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da107-92d3-4ca8-b8b7-44d49a383e2a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:22.388743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da107-92d3-4ca8-b8b7-44d49a383e2a None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:22.390359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 748/1485] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:22.396529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431513-7998-421f-a16b-57e481714735 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:22.397015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431513-7998-421f-a16b-57e481714735 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:22.397129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431513-7998-421f-a16b-57e481714735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:22.466666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-bf431513-7998-421f-a16b-57e481714735 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:22.467904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bf431513-7998-421f-a16b-57e481714735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:22.567525 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 738/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:22.865031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226da107-92d3-4ca8-b8b7-44d49a383e2a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:22.866586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-226da107-92d3-4ca8-b8b7-44d49a383e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:22.973251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226da107-92d3-4ca8-b8b7-44d49a383e2a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:23.047810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:23.049311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7f2f-7aa6-498c-b5ad-64309cb0ccb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:23.049919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7f2f-7aa6-498c-b5ad-64309cb0ccb2 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:23.050278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7f2f-7aa6-498c-b5ad-64309cb0ccb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:23.050578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7f2f-7aa6-498c-b5ad-64309cb0ccb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:23.052184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ae8c7f2f-7aa6-498c-b5ad-64309cb0ccb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:23.164293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7f2f-7aa6-498c-b5ad-64309cb0ccb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:23.165776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7f2f-7aa6-498c-b5ad-64309cb0ccb2 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:23.186200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 749/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:48:25.475510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ef12f-542b-46b5-a13c-bf51c992a8ba None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:25.476124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ef12f-542b-46b5-a13c-bf51c992a8ba None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:25.476124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ef12f-542b-46b5-a13c-bf51c992a8ba None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:25.477428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 739/1488] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:25.483248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4fe9f-fe25-4383-9d3d-a3c31f074f58 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:25.483746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4fe9f-fe25-4383-9d3d-a3c31f074f58 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:25.483856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4fe9f-fe25-4383-9d3d-a3c31f074f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:25.554170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-68d4fe9f-fe25-4383-9d3d-a3c31f074f58 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:25.555417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-68d4fe9f-fe25-4383-9d3d-a3c31f074f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:25.608684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:25.655887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 750/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:25.946389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ef12f-542b-46b5-a13c-bf51c992a8ba None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:25.948297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-042ef12f-542b-46b5-a13c-bf51c992a8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:25.983634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:26.058687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ef12f-542b-46b5-a13c-bf51c992a8ba None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:26.138729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4cec9-8975-48be-987f-354c6bafb30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:26.139346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4cec9-8975-48be-987f-354c6bafb30b service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:26.139628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4cec9-8975-48be-987f-354c6bafb30b service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:26.139904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4cec9-8975-48be-987f-354c6bafb30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:26.142192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-06c4cec9-8975-48be-987f-354c6bafb30b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:26.225306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:26.254606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4cec9-8975-48be-987f-354c6bafb30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:26.256032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4cec9-8975-48be-987f-354c6bafb30b service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:26.274474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 740/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:48:28.951027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e497e11-357b-4b97-a3d9-ddc8e28c8f26 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:28.951776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e497e11-357b-4b97-a3d9-ddc8e28c8f26 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:28.951776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e497e11-357b-4b97-a3d9-ddc8e28c8f26 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:28.952885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 751/1491] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 13:48:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:28.960630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e815b-813c-4b97-86ed-5e984f9244ff None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:28.961337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e815b-813c-4b97-86ed-5e984f9244ff None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:28.961337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e815b-813c-4b97-86ed-5e984f9244ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:29.035675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-eb5e815b-813c-4b97-86ed-5e984f9244ff None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:29.036914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-eb5e815b-813c-4b97-86ed-5e984f9244ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:29.136249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 741/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:29.450340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e497e11-357b-4b97-a3d9-ddc8e28c8f26 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:29.451881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1e497e11-357b-4b97-a3d9-ddc8e28c8f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:29.557465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e497e11-357b-4b97-a3d9-ddc8e28c8f26 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:29.631855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd316-b5c9-4f62-a940-93d0f7d9cc45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:29.632839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd316-b5c9-4f62-a940-93d0f7d9cc45 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:29.633376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd316-b5c9-4f62-a940-93d0f7d9cc45 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:29.633649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd316-b5c9-4f62-a940-93d0f7d9cc45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:29.635394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3a9dd316-b5c9-4f62-a940-93d0f7d9cc45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:29.644757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:29.745463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dd316-b5c9-4f62-a940-93d0f7d9cc45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:29.746844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dd316-b5c9-4f62-a940-93d0f7d9cc45 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:29.765168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 752/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:48:39.134733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=82 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:39.762900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=50 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:53.132078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b16a8-fc26-46ce-9cab-ddc81d63c36c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:53.132910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b16a8-fc26-46ce-9cab-ddc81d63c36c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:53.132910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b16a8-fc26-46ce-9cab-ddc81d63c36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:53.218659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-499b16a8-fc26-46ce-9cab-ddc81d63c36c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:49:53.219926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-499b16a8-fc26-46ce-9cab-ddc81d63c36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:53.316070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 742/1494] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:49:53.323802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d17e1-a9ab-4f39-9df6-3be4132e506a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:53.324306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d17e1-a9ab-4f39-9df6-3be4132e506a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:53.324415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d17e1-a9ab-4f39-9df6-3be4132e506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:53.398344 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-680d17e1-a9ab-4f39-9df6-3be4132e506a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:49:53.399550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-680d17e1-a9ab-4f39-9df6-3be4132e506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:53.496151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 753/1495] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:49:53.501893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b16a8-fc26-46ce-9cab-ddc81d63c36c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:53.503209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-499b16a8-fc26-46ce-9cab-ddc81d63c36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:53.515534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b16a8-fc26-46ce-9cab-ddc81d63c36c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:53.580251 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:53.616266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:53.616870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:53.617148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:53.617416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:53.618052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:49:53.618375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:53.619372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING py.warnings [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:49:53.619372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: warnings.warn(msg) Sep 08 13:49:53.619372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: Sep 08 13:49:53.620250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:53.632529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 743/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:49:53.641088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d17e1-a9ab-4f39-9df6-3be4132e506a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:53.650579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d17e1-a9ab-4f39-9df6-3be4132e506a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:53.712180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:53.745384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:53.745987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:53.746296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:53.746563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:53.747449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:53.749182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING py.warnings [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:49:53.749182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: warnings.warn(msg) Sep 08 13:49:53.749182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: Sep 08 13:49:53.750409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:53.872399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 754/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:49:53 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:49:53.876850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:53.878457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:53.912411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45698c31-286d-46e9-b531-bf40384ab26a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:54.007647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:54.008261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:54.008529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:54.008809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:54.009438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:54.010471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING py.warnings [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:49:54.010471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: warnings.warn(msg) Sep 08 13:49:54.010471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: Sep 08 13:49:54.011675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:54.117473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 744/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:49:53 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:49:54.121644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:54.123525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:54.185886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6979f-47da-46ec-8e9b-90fac6aba39a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:54.270598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:54.282648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:54.283255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:54.283527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:54.283791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:54.284427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:54.285457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING py.warnings [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:49:54.285457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: warnings.warn(msg) Sep 08 13:49:54.285457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: Sep 08 13:49:54.286419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:54.299189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 755/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:49:54.303499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:54.305096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:54.316479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2bd88-a6bd-44fc-ba69-4791eede7f0b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:54.341789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:54.412978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cabfe5-c418-43f6-94b8-e8b96be5b226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:54.413509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cabfe5-c418-43f6-94b8-e8b96be5b226 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:54.413790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cabfe5-c418-43f6-94b8-e8b96be5b226 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:54.414100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cabfe5-c418-43f6-94b8-e8b96be5b226 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:54.414777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cabfe5-c418-43f6-94b8-e8b96be5b226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:54.415797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING py.warnings [None req-15cabfe5-c418-43f6-94b8-e8b96be5b226 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:49:54.415797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: warnings.warn(msg) Sep 08 13:49:54.415797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: Sep 08 13:49:54.416700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cabfe5-c418-43f6-94b8-e8b96be5b226 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:54.429294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 745/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:49:54.433691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:54.435251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:54.469600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7181a-a879-489a-9538-cdded4a7b567 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:54.562521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c0baa-44f1-4bbd-bb3e-0c146a4351be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:54.564571 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c0baa-44f1-4bbd-bb3e-0c146a4351be admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:54.564571 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c0baa-44f1-4bbd-bb3e-0c146a4351be admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:54.564571 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c0baa-44f1-4bbd-bb3e-0c146a4351be admin admin] PATH_INFO: `/v3/projects/fbc687ae0d054d14868fa477526667ea/users/3ad3ce135a79424286c1d69540e8d203/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:54.587893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c0baa-44f1-4bbd-bb3e-0c146a4351be admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc687ae0d054d14868fa477526667ea, user_id=3ad3ce135a79424286c1d69540e8d203, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:54.588953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING py.warnings [None req-b67c0baa-44f1-4bbd-bb3e-0c146a4351be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:49:54.588953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: warnings.warn(msg) Sep 08 13:49:54.588953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: Sep 08 13:49:54.591042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c0baa-44f1-4bbd-bb3e-0c146a4351be admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:54.692735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 756/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:49:54 2022] PUT /identity/v3/projects/fbc687ae0d054d14868fa477526667ea/users/3ad3ce135a79424286c1d69540e8d203/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:49:54.705659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77901663-7c99-4ba5-8287-09fe67c42414 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:54.705950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77901663-7c99-4ba5-8287-09fe67c42414 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:54.706276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77901663-7c99-4ba5-8287-09fe67c42414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:54.763554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-77901663-7c99-4ba5-8287-09fe67c42414 None None] MFA Rules not processed for user `3ad3ce135a79424286c1d69540e8d203`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:49:54.764842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-77901663-7c99-4ba5-8287-09fe67c42414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:54.848254 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 746/1502] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:49:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:49:54.862347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce572a-61d1-490d-85af-c1f2a56db445 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:54.862640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce572a-61d1-490d-85af-c1f2a56db445 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:54.862921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce572a-61d1-490d-85af-c1f2a56db445 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:54.864266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 757/1503] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 8 13:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:49:54.869355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593bff3-3f4a-437b-9f61-8bab6fd57ca2 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:54.869651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593bff3-3f4a-437b-9f61-8bab6fd57ca2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:54.869928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593bff3-3f4a-437b-9f61-8bab6fd57ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:54.902419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:54.980953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b593bff3-3f4a-437b-9f61-8bab6fd57ca2 None None] MFA Rules not processed for user `817a190d001b40a2afee944f19270aee`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:49:54.982163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b593bff3-3f4a-437b-9f61-8bab6fd57ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:55.059562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 747/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 8 13:49:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:49:55.068642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce572a-61d1-490d-85af-c1f2a56db445 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:55.071343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-70ce572a-61d1-490d-85af-c1f2a56db445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:55.128784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:55.183464 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce572a-61d1-490d-85af-c1f2a56db445 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:55.256830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:55.257351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:55.257624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:55.257893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:55.259654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:55.363989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:55.365438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:55.368904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:55.383142 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 758/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:49:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:49:55.432449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8986a74-e55e-4bc9-9b89-b470ca8cff57 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:55.432919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8986a74-e55e-4bc9-9b89-b470ca8cff57 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:55.433028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8986a74-e55e-4bc9-9b89-b470ca8cff57 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:55.434376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 748/1506] 10.222.0.25 () {52 vars in 830 bytes} [Thu Sep 8 13:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:49:55.437964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:55.439269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:55.538411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46ccfd-4722-43f2-98b1-ddbc3b519434 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:55.650240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:55.650813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:55.651092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:55.651364 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] PATH_INFO: `/v3/users/3ad3ce135a79424286c1d69540e8d203` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:55.665824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad3ce135a79424286c1d69540e8d203)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:55.666899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING py.warnings [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:49:55.666899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: warnings.warn(msg) Sep 08 13:49:55.666899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: Sep 08 13:49:55.668126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:55.678205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:55.847952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad3ce135a79424286c1d69540e8d203'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:55.893782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad3ce135a79424286c1d69540e8d203'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:55.894278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad3ce135a79424286c1d69540e8d203'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:55.918766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad3ce135a79424286c1d69540e8d203'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:55.933584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad3ce135a79424286c1d69540e8d203'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:55.939927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 759/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:49:55 2022] DELETE /identity/v3/users/3ad3ce135a79424286c1d69540e8d203 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:49:55.944319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897a2c-129f-4330-bd4f-d9b995615e14 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:55.944633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897a2c-129f-4330-bd4f-d9b995615e14 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:55.944943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897a2c-129f-4330-bd4f-d9b995615e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:56.004499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-56897a2c-129f-4330-bd4f-d9b995615e14 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:49:56.005717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-56897a2c-129f-4330-bd4f-d9b995615e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:56.081955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:56.093966 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 749/1508] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:49:56.103908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:56.105092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:56.208894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4509414-798b-4cd8-b976-89f3d6bcd1f8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:56.271135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:49:56.290998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c85cae-82bc-4eda-af98-71a5dbad4c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:56.291643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c85cae-82bc-4eda-af98-71a5dbad4c0b service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:56.291923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c85cae-82bc-4eda-af98-71a5dbad4c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:56.292198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c85cae-82bc-4eda-af98-71a5dbad4c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:56.293806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-84c85cae-82bc-4eda-af98-71a5dbad4c0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:56.410778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c85cae-82bc-4eda-af98-71a5dbad4c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:56.412141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c85cae-82bc-4eda-af98-71a5dbad4c0b service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:56.431312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 760/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:49:56.947462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897a2c-129f-4330-bd4f-d9b995615e14 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:49:56.948705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-56897a2c-129f-4330-bd4f-d9b995615e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:49:56.960778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897a2c-129f-4330-bd4f-d9b995615e14 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:49:57.067754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:49:57.068393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:49:57.068674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:49:57.068949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] PATH_INFO: `/v3/projects/fbc687ae0d054d14868fa477526667ea` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:49:57.079528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc687ae0d054d14868fa477526667ea)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:49:57.080547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING py.warnings [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:49:57.080547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: warnings.warn(msg) Sep 08 13:49:57.080547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: Sep 08 13:49:57.081774 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:49:57.180109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc687ae0d054d14868fa477526667ea', 'request_id': 'req-4cca0adc-8b01-463b-b8b9-d0b177894c9f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:57.196350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc687ae0d054d14868fa477526667ea', 'request_id': 'req-4cca0adc-8b01-463b-b8b9-d0b177894c9f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:57.277313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc687ae0d054d14868fa477526667ea', 'request_id': 'req-4cca0adc-8b01-463b-b8b9-d0b177894c9f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:57.301543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] The token cache is being invalidate because project fbc687ae0d054d14868fa477526667ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:49:57.301841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:57.302403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4cca0adc-8b01-463b-b8b9-d0b177894c9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:49:57.303726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 750/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:49:56 2022] DELETE /identity/v3/projects/fbc687ae0d054d14868fa477526667ea => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:50:06.428981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=62 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:50:07.299635 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:30.152937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0562c-ba61-4566-9da4-c01983b07f84 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:30.153532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0562c-ba61-4566-9da4-c01983b07f84 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:30.153532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0562c-ba61-4566-9da4-c01983b07f84 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:30.154850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:30.159669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58972006-51ed-43ba-b633-6ff6e983d2b2 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:30.160131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58972006-51ed-43ba-b633-6ff6e983d2b2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:30.160243 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58972006-51ed-43ba-b633-6ff6e983d2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:30.241112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-58972006-51ed-43ba-b633-6ff6e983d2b2 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:30.242369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-58972006-51ed-43ba-b633-6ff6e983d2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:30.351708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:51:30.508213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0562c-ba61-4566-9da4-c01983b07f84 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:30.511498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5ee0562c-ba61-4566-9da4-c01983b07f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:30.630665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0562c-ba61-4566-9da4-c01983b07f84 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:30.636718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc97862-b2f7-41cb-9be1-08f919467dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:30.637280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97862-b2f7-41cb-9be1-08f919467dc7 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:30.637566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97862-b2f7-41cb-9be1-08f919467dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:30.637844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97862-b2f7-41cb-9be1-08f919467dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:30.646249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97862-b2f7-41cb-9be1-08f919467dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:30.647848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97862-b2f7-41cb-9be1-08f919467dc7 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:30.657953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 762/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:31.759171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651a0b1-81ed-4854-953a-c4d5126b673a None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:31.759750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651a0b1-81ed-4854-953a-c4d5126b673a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:31.759750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651a0b1-81ed-4854-953a-c4d5126b673a None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:31.761088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:31.768279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2d33-32e3-4c83-9b5c-e133e3ed8987 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:31.768811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2d33-32e3-4c83-9b5c-e133e3ed8987 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:31.768811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2d33-32e3-4c83-9b5c-e133e3ed8987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:31.789643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-273f2d33-32e3-4c83-9b5c-e133e3ed8987 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:31.790904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-273f2d33-32e3-4c83-9b5c-e133e3ed8987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:31.798974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:31.953719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651a0b1-81ed-4854-953a-c4d5126b673a None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:31.969348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651a0b1-81ed-4854-953a-c4d5126b673a None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:31.975132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65179921-2892-4093-bf4f-deedc3550069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:31.975751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179921-2892-4093-bf4f-deedc3550069 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:31.976042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179921-2892-4093-bf4f-deedc3550069 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:31.976465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179921-2892-4093-bf4f-deedc3550069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:31.987988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65179921-2892-4093-bf4f-deedc3550069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:31.990183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65179921-2892-4093-bf4f-deedc3550069 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:32.002042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 753/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:32.561502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58d0f-e832-45dd-b0ac-b515a6c5acb1 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:32.561784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58d0f-e832-45dd-b0ac-b515a6c5acb1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:32.562054 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58d0f-e832-45dd-b0ac-b515a6c5acb1 None None] PATH_INFO: `/v3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:32.563352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 764/1517] 10.222.0.16 () {54 vars in 925 bytes} [Thu Sep 8 13:51:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:51:32.571025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801bf7-2f58-456e-a67f-2409351c88d5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:32.571328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801bf7-2f58-456e-a67f-2409351c88d5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:32.571688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801bf7-2f58-456e-a67f-2409351c88d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:32.634452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e7801bf7-2f58-456e-a67f-2409351c88d5 None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:32.635685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e7801bf7-2f58-456e-a67f-2409351c88d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:32.653126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:32.709210 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 754/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Sep 8 13:51:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:41.784678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=56 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:42.706387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=23 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:43.391153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025b342-efb1-4ff4-9307-e7225ba4ca58 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:43.391821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025b342-efb1-4ff4-9307-e7225ba4ca58 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:43.391821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025b342-efb1-4ff4-9307-e7225ba4ca58 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:43.392911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 765/1519] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 8 13:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:43.398412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c318a-b982-4139-92f5-a1a06ec4d98c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:43.398704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c318a-b982-4139-92f5-a1a06ec4d98c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:43.400982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c318a-b982-4139-92f5-a1a06ec4d98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:43.419112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e70c318a-b982-4139-92f5-a1a06ec4d98c None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:43.420370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e70c318a-b982-4139-92f5-a1a06ec4d98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:43.429391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 755/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 8 13:51:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:43.437866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025b342-efb1-4ff4-9307-e7225ba4ca58 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:43.452487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025b342-efb1-4ff4-9307-e7225ba4ca58 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:43.457593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c95640-877d-456a-b33e-7f1f7e773432 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:43.458219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c95640-877d-456a-b33e-7f1f7e773432 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:43.458502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c95640-877d-456a-b33e-7f1f7e773432 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:43.458778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c95640-877d-456a-b33e-7f1f7e773432 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:43.467592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c95640-877d-456a-b33e-7f1f7e773432 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:43.470167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c95640-877d-456a-b33e-7f1f7e773432 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:43.480399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 766/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:43.518299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bdd27-c33d-4a47-9648-dd9dd23372d2 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:43.518588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bdd27-c33d-4a47-9648-dd9dd23372d2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:43.518870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bdd27-c33d-4a47-9648-dd9dd23372d2 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:43.520287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:43.526640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453197e-d092-4b11-90a8-2073957a6e15 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:43.526926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453197e-d092-4b11-90a8-2073957a6e15 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:43.527200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453197e-d092-4b11-90a8-2073957a6e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:43.547820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0453197e-d092-4b11-90a8-2073957a6e15 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:43.548913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0453197e-d092-4b11-90a8-2073957a6e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:43.556913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:43.715748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bdd27-c33d-4a47-9648-dd9dd23372d2 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:43.730190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bdd27-c33d-4a47-9648-dd9dd23372d2 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:43.736283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f592b5-926d-4b51-9938-b29a1997f0f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:43.744192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f592b5-926d-4b51-9938-b29a1997f0f8 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:43.744192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f592b5-926d-4b51-9938-b29a1997f0f8 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:43.744192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f592b5-926d-4b51-9938-b29a1997f0f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:43.745637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f592b5-926d-4b51-9938-b29a1997f0f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:43.747919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f592b5-926d-4b51-9938-b29a1997f0f8 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:43.757537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 757/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:44.843233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998cca1-63be-4081-96c5-fee873feb853 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:44.843819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998cca1-63be-4081-96c5-fee873feb853 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:44.843819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998cca1-63be-4081-96c5-fee873feb853 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:44.845131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:44.851413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b57c-b494-48e9-8394-9cdead8c28c0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:44.851834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b57c-b494-48e9-8394-9cdead8c28c0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:44.851990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b57c-b494-48e9-8394-9cdead8c28c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:44.871852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-4897b57c-b494-48e9-8394-9cdead8c28c0 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:44.873097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4897b57c-b494-48e9-8394-9cdead8c28c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:44.881488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:45.034671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998cca1-63be-4081-96c5-fee873feb853 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:45.049155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998cca1-63be-4081-96c5-fee873feb853 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:45.054518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78630ec5-3a3b-40f2-a6d0-04467ce64a69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:45.055111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78630ec5-3a3b-40f2-a6d0-04467ce64a69 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:45.055387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78630ec5-3a3b-40f2-a6d0-04467ce64a69 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:45.055682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78630ec5-3a3b-40f2-a6d0-04467ce64a69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:45.065248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78630ec5-3a3b-40f2-a6d0-04467ce64a69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:45.066868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78630ec5-3a3b-40f2-a6d0-04467ce64a69 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:45.076627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 769/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:49.255649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d1fa5-c6c6-4e67-9542-e572c44aeee9 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:49.256293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d1fa5-c6c6-4e67-9542-e572c44aeee9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:49.256578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d1fa5-c6c6-4e67-9542-e572c44aeee9 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:49.258803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:49.271960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8d897-a4ae-4411-aec7-1d3bb8926f23 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:49.272600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8d897-a4ae-4411-aec7-1d3bb8926f23 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:49.272600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8d897-a4ae-4411-aec7-1d3bb8926f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:49.299075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-45f8d897-a4ae-4411-aec7-1d3bb8926f23 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:49.301507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-45f8d897-a4ae-4411-aec7-1d3bb8926f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:49.322228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:51:49.626549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d1fa5-c6c6-4e67-9542-e572c44aeee9 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:49.628288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fc9d1fa5-c6c6-4e67-9542-e572c44aeee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:49.709932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d1fa5-c6c6-4e67-9542-e572c44aeee9 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:49.713762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e056c6-f8f3-40d2-9d44-8e8cbc321636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:49.714405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e056c6-f8f3-40d2-9d44-8e8cbc321636 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:49.714681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e056c6-f8f3-40d2-9d44-8e8cbc321636 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:49.714957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e056c6-f8f3-40d2-9d44-8e8cbc321636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:49.725495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e056c6-f8f3-40d2-9d44-8e8cbc321636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:49.726994 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e056c6-f8f3-40d2-9d44-8e8cbc321636 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:49.736619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 760/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:51.893734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbc890-e012-49b4-a05c-20181afd98bf None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:51.894280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbc890-e012-49b4-a05c-20181afd98bf None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:51.894280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbc890-e012-49b4-a05c-20181afd98bf None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:51.895554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:51.900749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b88f4-4fb9-4cea-8bcf-245f4b7bb671 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:51.901191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b88f4-4fb9-4cea-8bcf-245f4b7bb671 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:51.901305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b88f4-4fb9-4cea-8bcf-245f4b7bb671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:51.921275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0b8b88f4-4fb9-4cea-8bcf-245f4b7bb671 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:51.922545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0b8b88f4-4fb9-4cea-8bcf-245f4b7bb671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:51.930435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:51:52.220669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbc890-e012-49b4-a05c-20181afd98bf None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:52.231907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbc890-e012-49b4-a05c-20181afd98bf None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:52.235471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0e9f4-2d81-4df0-b482-52468bc21efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:52.236063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e9f4-2d81-4df0-b482-52468bc21efc service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:52.236343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e9f4-2d81-4df0-b482-52468bc21efc service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:52.236616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e9f4-2d81-4df0-b482-52468bc21efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:52.245358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0e9f4-2d81-4df0-b482-52468bc21efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:52.247485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0e9f4-2d81-4df0-b482-52468bc21efc service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:52.256972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 772/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:56.204898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5592e-6781-4183-b762-d8d187af19c5 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:56.205487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5592e-6781-4183-b762-d8d187af19c5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:56.205487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5592e-6781-4183-b762-d8d187af19c5 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:56.207422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:56.213687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c0d13-29ef-4941-9599-b9237a90a986 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:56.214211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c0d13-29ef-4941-9599-b9237a90a986 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:56.214326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c0d13-29ef-4941-9599-b9237a90a986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:56.237277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-b92c0d13-29ef-4941-9599-b9237a90a986 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:56.238530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b92c0d13-29ef-4941-9599-b9237a90a986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:56.246855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:56.557914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5592e-6781-4183-b762-d8d187af19c5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:56.560908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-60a5592e-6781-4183-b762-d8d187af19c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:56.648062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5592e-6781-4183-b762-d8d187af19c5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:56.652063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392c8c6-a67f-4704-b4a5-057f2b442845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:56.653533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392c8c6-a67f-4704-b4a5-057f2b442845 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:56.653792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392c8c6-a67f-4704-b4a5-057f2b442845 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:56.654131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392c8c6-a67f-4704-b4a5-057f2b442845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:56.663452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392c8c6-a67f-4704-b4a5-057f2b442845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:56.664878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392c8c6-a67f-4704-b4a5-057f2b442845 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:56.674096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 763/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:58.857781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ce4ef-321b-4ec6-953e-8304897b19b6 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:58.858386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ce4ef-321b-4ec6-953e-8304897b19b6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:58.858386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ce4ef-321b-4ec6-953e-8304897b19b6 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:58.859744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:51:58.865308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ab05a-d5b4-4bbf-9b6f-6e72959d6117 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:58.865759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ab05a-d5b4-4bbf-9b6f-6e72959d6117 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:58.865897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ab05a-d5b4-4bbf-9b6f-6e72959d6117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:58.887290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-171ab05a-d5b4-4bbf-9b6f-6e72959d6117 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:58.888563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-171ab05a-d5b4-4bbf-9b6f-6e72959d6117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:58.896483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:59.202998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ce4ef-321b-4ec6-953e-8304897b19b6 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:59.204809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b29ce4ef-321b-4ec6-953e-8304897b19b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:59.217749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ce4ef-321b-4ec6-953e-8304897b19b6 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:59.221304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9cf7e-25bf-4539-bc58-1cca26f5bb89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:59.221926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9cf7e-25bf-4539-bc58-1cca26f5bb89 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:59.222231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9cf7e-25bf-4539-bc58-1cca26f5bb89 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:59.222513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9cf7e-25bf-4539-bc58-1cca26f5bb89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:59.232311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9cf7e-25bf-4539-bc58-1cca26f5bb89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:59.233740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9cf7e-25bf-4539-bc58-1cca26f5bb89 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:59.243349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 775/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:08.881934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=88 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:09.239030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=33 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:25.361559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454daf-05f3-4f67-9637-4e652df8022c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.362203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454daf-05f3-4f67-9637-4e652df8022c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.362508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454daf-05f3-4f67-9637-4e652df8022c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.403163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-67454daf-05f3-4f67-9637-4e652df8022c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:25.404763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-67454daf-05f3-4f67-9637-4e652df8022c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:25.418762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 765/1540] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:25.427617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be406df-2ebe-40f4-a59a-3b818fb8f52b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.428191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be406df-2ebe-40f4-a59a-3b818fb8f52b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.428191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be406df-2ebe-40f4-a59a-3b818fb8f52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.454590 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-9be406df-2ebe-40f4-a59a-3b818fb8f52b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:25.455948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9be406df-2ebe-40f4-a59a-3b818fb8f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:25.469203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 776/1541] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:25.474229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67454daf-05f3-4f67-9637-4e652df8022c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:25.485761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67454daf-05f3-4f67-9637-4e652df8022c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:25.494231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7482ef9-cdcb-4783-9ad5-e841e7a8785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:25.494880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7482ef9-cdcb-4783-9ad5-e841e7a8785e admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.495327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7482ef9-cdcb-4783-9ad5-e841e7a8785e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.495611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7482ef9-cdcb-4783-9ad5-e841e7a8785e admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.496886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7482ef9-cdcb-4783-9ad5-e841e7a8785e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:25.497199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7482ef9-cdcb-4783-9ad5-e841e7a8785e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:25.498932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7482ef9-cdcb-4783-9ad5-e841e7a8785e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:25.520469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 766/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:25.529343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be406df-2ebe-40f4-a59a-3b818fb8f52b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:25.539728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be406df-2ebe-40f4-a59a-3b818fb8f52b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:25.550877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af579f-2f7e-49e5-a070-b4727f96c1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:25.551526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af579f-2f7e-49e5-a070-b4727f96c1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.551799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af579f-2f7e-49e5-a070-b4727f96c1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.552071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af579f-2f7e-49e5-a070-b4727f96c1b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.552740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af579f-2f7e-49e5-a070-b4727f96c1b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:25.554765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af579f-2f7e-49e5-a070-b4727f96c1b8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:25.608664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aecb13-6857-42bd-88dc-a1f9dcb697f1 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.622301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aecb13-6857-42bd-88dc-a1f9dcb697f1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.622301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aecb13-6857-42bd-88dc-a1f9dcb697f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.640025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b1aecb13-6857-42bd-88dc-a1f9dcb697f1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:25.641257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b1aecb13-6857-42bd-88dc-a1f9dcb697f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:25.651244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 767/1543] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:25.663322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b9433-8153-4842-92ae-0292f765c807 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.663664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b9433-8153-4842-92ae-0292f765c807 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.663882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b9433-8153-4842-92ae-0292f765c807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.697292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-4c6b9433-8153-4842-92ae-0292f765c807 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:25.699112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4c6b9433-8153-4842-92ae-0292f765c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:25.716979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 768/1544] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:25.724659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b9433-8153-4842-92ae-0292f765c807 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:25.736695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b9433-8153-4842-92ae-0292f765c807 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:25.748114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce701353-2ca2-4b65-b3ac-2262dea0de2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:25.749166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701353-2ca2-4b65-b3ac-2262dea0de2f admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.749746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701353-2ca2-4b65-b3ac-2262dea0de2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.750123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701353-2ca2-4b65-b3ac-2262dea0de2f admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.750923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce701353-2ca2-4b65-b3ac-2262dea0de2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:25.751231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce701353-2ca2-4b65-b3ac-2262dea0de2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:25.753101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce701353-2ca2-4b65-b3ac-2262dea0de2f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:25.768686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 769/1545] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:53:25.769992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8743f1-085d-4e7e-acb9-c92dc2bf23a8 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.770378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8743f1-085d-4e7e-acb9-c92dc2bf23a8 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.770659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8743f1-085d-4e7e-acb9-c92dc2bf23a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.787900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 777/1546] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:25.788586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af579f-2f7e-49e5-a070-b4727f96c1b8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:25.824287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3d8743f1-085d-4e7e-acb9-c92dc2bf23a8 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:25.825993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3d8743f1-085d-4e7e-acb9-c92dc2bf23a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:25.901385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af579f-2f7e-49e5-a070-b4727f96c1b8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:25.909823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a0010-3259-414c-a42a-c6c6a7a44e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:25.910861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0010-3259-414c-a42a-c6c6a7a44e14 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.911143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0010-3259-414c-a42a-c6c6a7a44e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.911419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0010-3259-414c-a42a-c6c6a7a44e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.912247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a0010-3259-414c-a42a-c6c6a7a44e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:25.912369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 770/1547] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:25.912899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8743f1-085d-4e7e-acb9-c92dc2bf23a8 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:25.914077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a0010-3259-414c-a42a-c6c6a7a44e14 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:25.923406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8743f1-085d-4e7e-acb9-c92dc2bf23a8 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:25.930995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8080cef-27c1-4809-a6d5-81ebe6b1b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:25.931608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080cef-27c1-4809-a6d5-81ebe6b1b011 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.931885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080cef-27c1-4809-a6d5-81ebe6b1b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.932157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080cef-27c1-4809-a6d5-81ebe6b1b011 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.932798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8080cef-27c1-4809-a6d5-81ebe6b1b011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:25.934808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8080cef-27c1-4809-a6d5-81ebe6b1b011 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.039178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 778/1548] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/projects => generated 373 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:26.040639 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c59251-e7a3-4a56-a73c-90ff2dbb5a94 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.040934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c59251-e7a3-4a56-a73c-90ff2dbb5a94 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.041211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c59251-e7a3-4a56-a73c-90ff2dbb5a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.067656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 771/1549] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:26.071005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eab42-c20f-49f5-a033-b9a336146974 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.071005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eab42-c20f-49f5-a033-b9a336146974 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.071005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eab42-c20f-49f5-a033-b9a336146974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.072936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-45c59251-e7a3-4a56-a73c-90ff2dbb5a94 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.074826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-45c59251-e7a3-4a56-a73c-90ff2dbb5a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.096292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-903eab42-c20f-49f5-a033-b9a336146974 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.098032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-903eab42-c20f-49f5-a033-b9a336146974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.138082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 779/1550] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.139501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7476b19-d9e4-464c-9725-2c71c95728bd None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.139788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7476b19-d9e4-464c-9725-2c71c95728bd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.140064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7476b19-d9e4-464c-9725-2c71c95728bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.167535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d7476b19-d9e4-464c-9725-2c71c95728bd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.167535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d7476b19-d9e4-464c-9725-2c71c95728bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.169755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 780/1551] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.170993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ad1a4-bb1e-48ae-b7a5-5b9db57950e3 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.171278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ad1a4-bb1e-48ae-b7a5-5b9db57950e3 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.171554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ad1a4-bb1e-48ae-b7a5-5b9db57950e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.176709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 772/1552] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.183871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782b3df-25d2-4439-850f-7cd6057fdb34 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.183871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782b3df-25d2-4439-850f-7cd6057fdb34 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.183871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782b3df-25d2-4439-850f-7cd6057fdb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.192183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0f6ad1a4-bb1e-48ae-b7a5-5b9db57950e3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.193395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0f6ad1a4-bb1e-48ae-b7a5-5b9db57950e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.198096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3782b3df-25d2-4439-850f-7cd6057fdb34 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.199819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3782b3df-25d2-4439-850f-7cd6057fdb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.202009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 781/1553] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.203240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d34d5-6f77-41fe-8790-10db01f85b29 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.203520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d34d5-6f77-41fe-8790-10db01f85b29 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.203790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d34d5-6f77-41fe-8790-10db01f85b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.213362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 773/1554] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.214431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782b3df-25d2-4439-850f-7cd6057fdb34 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.224584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782b3df-25d2-4439-850f-7cd6057fdb34 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.228466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1f6d34d5-6f77-41fe-8790-10db01f85b29 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.229188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba399270-21a2-4442-a877-e149963b8a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.229672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1f6d34d5-6f77-41fe-8790-10db01f85b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.229788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba399270-21a2-4442-a877-e149963b8a49 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.230046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba399270-21a2-4442-a877-e149963b8a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.230413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba399270-21a2-4442-a877-e149963b8a49 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.231040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba399270-21a2-4442-a877-e149963b8a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.232939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba399270-21a2-4442-a877-e149963b8a49 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.238239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 782/1555] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.239026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d34d5-6f77-41fe-8790-10db01f85b29 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.251351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d34d5-6f77-41fe-8790-10db01f85b29 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.258419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1201239-895f-457d-82b4-214c7b113154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.265116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1201239-895f-457d-82b4-214c7b113154 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.265116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1201239-895f-457d-82b4-214c7b113154 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.265116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1201239-895f-457d-82b4-214c7b113154 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.265116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1201239-895f-457d-82b4-214c7b113154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.265116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1201239-895f-457d-82b4-214c7b113154 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.273660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 783/1556] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:26.274934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643ad4-e3ac-49d5-b644-c88d9a043b71 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.275212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643ad4-e3ac-49d5-b644-c88d9a043b71 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.275481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643ad4-e3ac-49d5-b644-c88d9a043b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.295053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-51643ad4-e3ac-49d5-b644-c88d9a043b71 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.299621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-51643ad4-e3ac-49d5-b644-c88d9a043b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.304080 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 784/1557] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.305219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355247b-d09c-44e0-a5a3-4eb8efff8099 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.305487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355247b-d09c-44e0-a5a3-4eb8efff8099 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.305759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355247b-d09c-44e0-a5a3-4eb8efff8099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.324878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0355247b-d09c-44e0-a5a3-4eb8efff8099 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.329968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 774/1558] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:26.330108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0355247b-d09c-44e0-a5a3-4eb8efff8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.330245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c898c-b6cc-4c3b-a34e-907e23ad27a3 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.330245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c898c-b6cc-4c3b-a34e-907e23ad27a3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.330245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c898c-b6cc-4c3b-a34e-907e23ad27a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.335470 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 785/1559] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.336777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26f009-616d-4911-bfd4-a421200cb0cb None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.337132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26f009-616d-4911-bfd4-a421200cb0cb None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.337479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26f009-616d-4911-bfd4-a421200cb0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.347108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:26.350466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-081c898c-b6cc-4c3b-a34e-907e23ad27a3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.351780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-081c898c-b6cc-4c3b-a34e-907e23ad27a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.359223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-2b26f009-616d-4911-bfd4-a421200cb0cb None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.360412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2b26f009-616d-4911-bfd4-a421200cb0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.361671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 775/1560] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.362608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c898c-b6cc-4c3b-a34e-907e23ad27a3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.371243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 786/1561] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.372521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c898c-b6cc-4c3b-a34e-907e23ad27a3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.376170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efd2cd-8d43-4e4f-af63-d31f02fcf59a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.376461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efd2cd-8d43-4e4f-af63-d31f02fcf59a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.376741 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efd2cd-8d43-4e4f-af63-d31f02fcf59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.378557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.379330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.379617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.379943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.380551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:26.380859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.382580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.386422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:26.397032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 776/1562] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:26.397940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.398952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-95efd2cd-8d43-4e4f-af63-d31f02fcf59a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:26.400236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-95efd2cd-8d43-4e4f-af63-d31f02fcf59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:26.408790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926bfa1-421d-4346-95a7-9a09462cd8a6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.410526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 787/1563] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:26.412768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efd2cd-8d43-4e4f-af63-d31f02fcf59a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.413660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c125-b6dd-4b6f-9dd1-57080ae998c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.414431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c125-b6dd-4b6f-9dd1-57080ae998c0 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.414728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c125-b6dd-4b6f-9dd1-57080ae998c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.415007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c125-b6dd-4b6f-9dd1-57080ae998c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.415663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86c125-b6dd-4b6f-9dd1-57080ae998c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.417384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86c125-b6dd-4b6f-9dd1-57080ae998c0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.425784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efd2cd-8d43-4e4f-af63-d31f02fcf59a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.431285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 777/1564] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:26.431463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.432070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.432515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c125-b6dd-4b6f-9dd1-57080ae998c0 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.433236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.433548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.434269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:26.434653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.437966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING py.warnings [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:53:26.437966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: warnings.warn(msg) Sep 08 13:53:26.437966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: Sep 08 13:53:26.439920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.443758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c125-b6dd-4b6f-9dd1-57080ae998c0 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.448977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd08d6-da9b-4502-9416-9f8e891e92e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.450115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd08d6-da9b-4502-9416-9f8e891e92e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.450491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd08d6-da9b-4502-9416-9f8e891e92e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.450763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd08d6-da9b-4502-9416-9f8e891e92e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.451413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd08d6-da9b-4502-9416-9f8e891e92e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.453011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd08d6-da9b-4502-9416-9f8e891e92e5 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.460170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 788/1565] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:26.460170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.464236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 778/1566] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:26.465660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd08d6-da9b-4502-9416-9f8e891e92e5 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.476276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd08d6-da9b-4502-9416-9f8e891e92e5 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.476481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904f639-4468-46ea-b754-bdec54373b30 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.481348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.481819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.481955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.482838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.482950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.492404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.492553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.492671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:26.492770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.492856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.492952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:26.492952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.493138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.493223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.499215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 789/1567] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:26.500066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.505496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 779/1568] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:26.506603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.517843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df7cd-d68a-4d10-9c4a-b54da0145ee3 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.518026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85890-58f1-4d00-b9e9-5f81c1168248 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.522510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.523118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.523284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.523394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.523659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.524077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.524290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.524486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.524824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.525294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING py.warnings [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:53:26.525294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: warnings.warn(msg) Sep 08 13:53:26.525294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: Sep 08 13:53:26.525628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:26.525868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.526542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.527620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.613703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 790/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:26.614785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.625427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34501efd-185f-4440-86a2-9bf764823c8e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.630074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.630687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.631283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.631593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.632242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:26.632539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.634155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.647511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 791/1570] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:26.648457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.658552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44c07-c604-4a1e-ab24-2c3cacb24fb7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.663252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a625b42-8e44-4b78-b005-593ff7fbd73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.665642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a625b42-8e44-4b78-b005-593ff7fbd73a admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.665642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a625b42-8e44-4b78-b005-593ff7fbd73a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.665642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a625b42-8e44-4b78-b005-593ff7fbd73a admin admin] PATH_INFO: `/v3/projects/7711a822e7ef4fbda3d19219739b228e/users/9b91f50cffd149f29d30d68083f749dd/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.679058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a625b42-8e44-4b78-b005-593ff7fbd73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7711a822e7ef4fbda3d19219739b228e, user_id=9b91f50cffd149f29d30d68083f749dd, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.681670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a625b42-8e44-4b78-b005-593ff7fbd73a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.723654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 780/1571] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/projects => generated 381 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:26.723654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.759688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 792/1572] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:26 2022] PUT /identity/v3/projects/7711a822e7ef4fbda3d19219739b228e/users/9b91f50cffd149f29d30d68083f749dd/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:26.760870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a625b42-8e44-4b78-b005-593ff7fbd73a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.791684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce631fb-969b-4d3c-b1db-f45288c2aae9 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.832266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a625b42-8e44-4b78-b005-593ff7fbd73a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.837084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42248-4518-4d56-b689-6bef6f2d6c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.837696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42248-4518-4d56-b689-6bef6f2d6c55 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.837978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42248-4518-4d56-b689-6bef6f2d6c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.838396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42248-4518-4d56-b689-6bef6f2d6c55 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.839041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a42248-4518-4d56-b689-6bef6f2d6c55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.841038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a42248-4518-4d56-b689-6bef6f2d6c55 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.850736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed3666-60f5-437b-9a9c-06174ce10106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.851353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed3666-60f5-437b-9a9c-06174ce10106 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.851633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed3666-60f5-437b-9a9c-06174ce10106 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.851911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed3666-60f5-437b-9a9c-06174ce10106 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.852551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed3666-60f5-437b-9a9c-06174ce10106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.854147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed3666-60f5-437b-9a9c-06174ce10106 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.864661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 781/1573] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:26.865601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed3666-60f5-437b-9a9c-06174ce10106 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.875899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed3666-60f5-437b-9a9c-06174ce10106 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:26.881898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6e3c9-7b5a-4173-9ace-8772261c3618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:26.881898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e3c9-7b5a-4173-9ace-8772261c3618 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:26.881898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e3c9-7b5a-4173-9ace-8772261c3618 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:26.881898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e3c9-7b5a-4173-9ace-8772261c3618 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:26.882310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6e3c9-7b5a-4173-9ace-8772261c3618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:26.883967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6e3c9-7b5a-4173-9ace-8772261c3618 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:26.942836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 793/1574] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:26.943726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42248-4518-4d56-b689-6bef6f2d6c55 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.970902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 782/1575] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:26.982120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6e3c9-7b5a-4173-9ace-8772261c3618 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:26.991854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.006685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42248-4518-4d56-b689-6bef6f2d6c55 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.010881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.036013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6e3c9-7b5a-4173-9ace-8772261c3618 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.040740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a91771-4699-49dc-8e48-03dfd398ed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.041350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a91771-4699-49dc-8e48-03dfd398ed37 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.041629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a91771-4699-49dc-8e48-03dfd398ed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.041904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a91771-4699-49dc-8e48-03dfd398ed37 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.042651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a91771-4699-49dc-8e48-03dfd398ed37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.045076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a91771-4699-49dc-8e48-03dfd398ed37 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.063800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb2a41-125e-4472-bf24-2a4ebfd400a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.064414 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2a41-125e-4472-bf24-2a4ebfd400a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.064695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2a41-125e-4472-bf24-2a4ebfd400a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.064999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2a41-125e-4472-bf24-2a4ebfd400a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.065656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb2a41-125e-4472-bf24-2a4ebfd400a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.067590 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb2a41-125e-4472-bf24-2a4ebfd400a9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.170190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 783/1576] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.170190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a91771-4699-49dc-8e48-03dfd398ed37 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.217024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 794/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:26 2022] POST /identity/v3/projects => generated 365 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.217024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb2a41-125e-4472-bf24-2a4ebfd400a9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.228043 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a91771-4699-49dc-8e48-03dfd398ed37 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.273945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb2a41-125e-4472-bf24-2a4ebfd400a9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.279497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5c10d-a6b5-4d8f-8d6b-536c742c7861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.280107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c10d-a6b5-4d8f-8d6b-536c742c7861 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.280386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c10d-a6b5-4d8f-8d6b-536c742c7861 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.280654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c10d-a6b5-4d8f-8d6b-536c742c7861 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.281281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5c10d-a6b5-4d8f-8d6b-536c742c7861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.282318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING py.warnings [None req-5aa5c10d-a6b5-4d8f-8d6b-536c742c7861 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:53:27.282318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: warnings.warn(msg) Sep 08 13:53:27.282318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: Sep 08 13:53:27.283529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5c10d-a6b5-4d8f-8d6b-536c742c7861 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.286558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2f1ac-a02f-4800-8f63-97dc326be691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.292211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2f1ac-a02f-4800-8f63-97dc326be691 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.292211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2f1ac-a02f-4800-8f63-97dc326be691 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.292211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2f1ac-a02f-4800-8f63-97dc326be691 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.292211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2f1ac-a02f-4800-8f63-97dc326be691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.292211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2f1ac-a02f-4800-8f63-97dc326be691 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.404182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 795/1578] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.405555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ad1ac-6d9f-4986-af8d-4c519b9fc232 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.405841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ad1ac-6d9f-4986-af8d-4c519b9fc232 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.406115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ad1ac-6d9f-4986-af8d-4c519b9fc232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.437944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 784/1579] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/projects => generated 391 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.438551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ad2ad1ac-6d9f-4986-af8d-4c519b9fc232 None None] MFA Rules not processed for user `9b91f50cffd149f29d30d68083f749dd`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:27.438924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2f1ac-a02f-4800-8f63-97dc326be691 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.439030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ad2ad1ac-6d9f-4986-af8d-4c519b9fc232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:27.476852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.500293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2f1ac-a02f-4800-8f63-97dc326be691 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.504454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 796/1580] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:27.505506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ad1ac-6d9f-4986-af8d-4c519b9fc232 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.505918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.506702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.508802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.508802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] PATH_INFO: `/v3/projects/7538afd39b364087a38b64080f27ec58/users/bfc74593ee634c619b783f9daa20312c/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.518278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ad1ac-6d9f-4986-af8d-4c519b9fc232 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.523284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7538afd39b364087a38b64080f27ec58, user_id=bfc74593ee634c619b783f9daa20312c, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.523806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb905c84-d843-4686-8ccd-4e5381c9a7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.524316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING py.warnings [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:53:27.524316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: warnings.warn(msg) Sep 08 13:53:27.524316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: Sep 08 13:53:27.524579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb905c84-d843-4686-8ccd-4e5381c9a7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.524666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb905c84-d843-4686-8ccd-4e5381c9a7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.525349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb905c84-d843-4686-8ccd-4e5381c9a7f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.525542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb905c84-d843-4686-8ccd-4e5381c9a7f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.526297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.528278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb905c84-d843-4686-8ccd-4e5381c9a7f1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.539043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.598633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 785/1581] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:27 2022] PUT /identity/v3/projects/7538afd39b364087a38b64080f27ec58/users/bfc74593ee634c619b783f9daa20312c/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:27.604316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.633194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 797/1582] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.633194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb905c84-d843-4686-8ccd-4e5381c9a7f1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.642384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb905c84-d843-4686-8ccd-4e5381c9a7f1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.666956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72831fea-ddb0-452c-aa41-00bdb054a7c7 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.671858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e052d-5168-4a70-9eec-ab37934a29d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.672528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e052d-5168-4a70-9eec-ab37934a29d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.672819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e052d-5168-4a70-9eec-ab37934a29d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.673095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e052d-5168-4a70-9eec-ab37934a29d1 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.673757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e052d-5168-4a70-9eec-ab37934a29d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.675757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e052d-5168-4a70-9eec-ab37934a29d1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.703808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1aa2-d519-4d74-93ff-bb8ab4dc5b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.704412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1aa2-d519-4d74-93ff-bb8ab4dc5b66 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.704689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1aa2-d519-4d74-93ff-bb8ab4dc5b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.704959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1aa2-d519-4d74-93ff-bb8ab4dc5b66 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.705588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa1aa2-d519-4d74-93ff-bb8ab4dc5b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.707538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa1aa2-d519-4d74-93ff-bb8ab4dc5b66 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.759265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 786/1583] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.760618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e052d-5168-4a70-9eec-ab37934a29d1 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.770818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e052d-5168-4a70-9eec-ab37934a29d1 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.775578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d618-121f-4bba-b6a2-19926bdd42c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.776163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d618-121f-4bba-b6a2-19926bdd42c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.776443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d618-121f-4bba-b6a2-19926bdd42c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.776856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d618-121f-4bba-b6a2-19926bdd42c2 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.778058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127d618-121f-4bba-b6a2-19926bdd42c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.780002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127d618-121f-4bba-b6a2-19926bdd42c2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.818171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 798/1584] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.819698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1aa2-d519-4d74-93ff-bb8ab4dc5b66 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.831051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1aa2-d519-4d74-93ff-bb8ab4dc5b66 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.835903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eed365-73a6-4b12-a360-43a46a7db586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.836499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed365-73a6-4b12-a360-43a46a7db586 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.836779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed365-73a6-4b12-a360-43a46a7db586 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.837054 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed365-73a6-4b12-a360-43a46a7db586 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.837693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eed365-73a6-4b12-a360-43a46a7db586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.839339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eed365-73a6-4b12-a360-43a46a7db586 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.850368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 799/1585] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:27.851143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eed365-73a6-4b12-a360-43a46a7db586 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.861226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eed365-73a6-4b12-a360-43a46a7db586 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.865954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839f158-3537-47fa-8e52-da117c970aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.872027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839f158-3537-47fa-8e52-da117c970aaf admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.872027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839f158-3537-47fa-8e52-da117c970aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.872027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839f158-3537-47fa-8e52-da117c970aaf admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.872027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839f158-3537-47fa-8e52-da117c970aaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.872405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 787/1586] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.872490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839f158-3537-47fa-8e52-da117c970aaf admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.872576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47933275-2b1c-40f4-a05a-5c8dc6e599d0 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.872576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47933275-2b1c-40f4-a05a-5c8dc6e599d0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.872576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47933275-2b1c-40f4-a05a-5c8dc6e599d0 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.873240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 788/1587] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 8 13:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:27.875017 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b790c-0da4-4b5e-9b12-bf69a1db28ef None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.875302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b790c-0da4-4b5e-9b12-bf69a1db28ef None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.875731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b790c-0da4-4b5e-9b12-bf69a1db28ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.905022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0b0b790c-0da4-4b5e-9b12-bf69a1db28ef None None] MFA Rules not processed for user `bfc74593ee634c619b783f9daa20312c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:27.906314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0b0b790c-0da4-4b5e-9b12-bf69a1db28ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:27.948520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 800/1588] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:27.949360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839f158-3537-47fa-8e52-da117c970aaf admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.961471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839f158-3537-47fa-8e52-da117c970aaf admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.966199 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ab9e0-aa18-4088-8eda-33de54aedff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.966803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ab9e0-aa18-4088-8eda-33de54aedff2 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.967085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ab9e0-aa18-4088-8eda-33de54aedff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.967379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ab9e0-aa18-4088-8eda-33de54aedff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.968016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ab9e0-aa18-4088-8eda-33de54aedff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.969809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ab9e0-aa18-4088-8eda-33de54aedff2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.973734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 789/1589] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:27.974678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b790c-0da4-4b5e-9b12-bf69a1db28ef None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.984557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 801/1590] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:27.985531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ab9e0-aa18-4088-8eda-33de54aedff2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.991170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b790c-0da4-4b5e-9b12-bf69a1db28ef None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.997815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9939cb-8e8f-4fed-9f78-64ce4b78404d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.998550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9939cb-8e8f-4fed-9f78-64ce4b78404d admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.998677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ab9e0-aa18-4088-8eda-33de54aedff2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.998846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9939cb-8e8f-4fed-9f78-64ce4b78404d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.999380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9939cb-8e8f-4fed-9f78-64ce4b78404d admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.000601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9939cb-8e8f-4fed-9f78-64ce4b78404d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.003249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9939cb-8e8f-4fed-9f78-64ce4b78404d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.003955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc809863-ded0-41de-b6e2-5ef7447a8caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.004568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc809863-ded0-41de-b6e2-5ef7447a8caa admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.004875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc809863-ded0-41de-b6e2-5ef7447a8caa admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.005145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc809863-ded0-41de-b6e2-5ef7447a8caa admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.005788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc809863-ded0-41de-b6e2-5ef7447a8caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.007523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc809863-ded0-41de-b6e2-5ef7447a8caa admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.016943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 790/1591] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.018029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9939cb-8e8f-4fed-9f78-64ce4b78404d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.020486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 802/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.028166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc809863-ded0-41de-b6e2-5ef7447a8caa admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.034012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9939cb-8e8f-4fed-9f78-64ce4b78404d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.034241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc809863-ded0-41de-b6e2-5ef7447a8caa admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.038814 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e5dc6-cde9-48bf-b277-23efe4bf72de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.038993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c37154-aef8-4e3c-ad8d-c989a8474603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.039420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e5dc6-cde9-48bf-b277-23efe4bf72de admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.039591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c37154-aef8-4e3c-ad8d-c989a8474603 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.039770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e5dc6-cde9-48bf-b277-23efe4bf72de admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.039872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c37154-aef8-4e3c-ad8d-c989a8474603 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.040048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e5dc6-cde9-48bf-b277-23efe4bf72de admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.040199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c37154-aef8-4e3c-ad8d-c989a8474603 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.040823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e5dc6-cde9-48bf-b277-23efe4bf72de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.040965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c37154-aef8-4e3c-ad8d-c989a8474603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.042458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e5dc6-cde9-48bf-b277-23efe4bf72de admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.042597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c37154-aef8-4e3c-ad8d-c989a8474603 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.046104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.053261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 803/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.057243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1e63-4608-47c3-b535-ebee3f2129e2 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.057243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1e63-4608-47c3-b535-ebee3f2129e2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.057243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1e63-4608-47c3-b535-ebee3f2129e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.057499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 791/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.057499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c37154-aef8-4e3c-ad8d-c989a8474603 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.067214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c37154-aef8-4e3c-ad8d-c989a8474603 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.072425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9bf96-cdf0-4f26-a401-4eb0ae3fd362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.073164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9bf96-cdf0-4f26-a401-4eb0ae3fd362 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.073533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9bf96-cdf0-4f26-a401-4eb0ae3fd362 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.073727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9bf96-cdf0-4f26-a401-4eb0ae3fd362 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.074483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9bf96-cdf0-4f26-a401-4eb0ae3fd362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.076824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9bf96-cdf0-4f26-a401-4eb0ae3fd362 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.087084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 792/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.087084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9bf96-cdf0-4f26-a401-4eb0ae3fd362 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.099659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-58eb1e63-4608-47c3-b535-ebee3f2129e2 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:28.099863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9bf96-cdf0-4f26-a401-4eb0ae3fd362 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.101013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-58eb1e63-4608-47c3-b535-ebee3f2129e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.105194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116d623-52f5-4abd-b157-12c1ff2a6a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.106568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116d623-52f5-4abd-b157-12c1ff2a6a4c admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.108901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116d623-52f5-4abd-b157-12c1ff2a6a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.108901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116d623-52f5-4abd-b157-12c1ff2a6a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.108901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116d623-52f5-4abd-b157-12c1ff2a6a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.109339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116d623-52f5-4abd-b157-12c1ff2a6a4c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.119616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 793/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.123373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0fbd1-0808-4792-88f6-bfcd45be7a95 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.123373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0fbd1-0808-4792-88f6-bfcd45be7a95 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.123373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0fbd1-0808-4792-88f6-bfcd45be7a95 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.123373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 794/1597] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 8 13:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:28.123874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0fbd1-0808-4792-88f6-bfcd45be7a95 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.141782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0fbd1-0808-4792-88f6-bfcd45be7a95 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.148176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f9e36-c4b6-4c2d-8c50-68d579d1255c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.148798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9e36-c4b6-4c2d-8c50-68d579d1255c admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.149082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9e36-c4b6-4c2d-8c50-68d579d1255c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.149357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9e36-c4b6-4c2d-8c50-68d579d1255c admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.150265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f9e36-c4b6-4c2d-8c50-68d579d1255c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.152148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f9e36-c4b6-4c2d-8c50-68d579d1255c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.162863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.167587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 795/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.167587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f9e36-c4b6-4c2d-8c50-68d579d1255c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.175929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f9e36-c4b6-4c2d-8c50-68d579d1255c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.179499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 804/1599] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 8 13:53:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:28.180548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1978d8-4aff-4330-af2f-a26cf9eab426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.180688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb1e63-4608-47c3-b535-ebee3f2129e2 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.181159 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1978d8-4aff-4330-af2f-a26cf9eab426 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.181436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1978d8-4aff-4330-af2f-a26cf9eab426 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.181702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1978d8-4aff-4330-af2f-a26cf9eab426 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.182407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1978d8-4aff-4330-af2f-a26cf9eab426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.184321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1978d8-4aff-4330-af2f-a26cf9eab426 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.193929 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb1e63-4608-47c3-b535-ebee3f2129e2 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.196149 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 796/1600] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.196650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1978d8-4aff-4330-af2f-a26cf9eab426 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.199380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bff282-313f-4173-ae08-a9a64102f4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.200214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bff282-313f-4173-ae08-a9a64102f4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.200402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bff282-313f-4173-ae08-a9a64102f4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.200678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bff282-313f-4173-ae08-a9a64102f4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.201327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bff282-313f-4173-ae08-a9a64102f4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.203047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bff282-313f-4173-ae08-a9a64102f4e0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.207348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1978d8-4aff-4330-af2f-a26cf9eab426 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.212396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a78ba-077a-4c14-85fb-56185f049fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.213173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a78ba-077a-4c14-85fb-56185f049fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.213849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a78ba-077a-4c14-85fb-56185f049fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.213849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a78ba-077a-4c14-85fb-56185f049fb8 admin admin] PATH_INFO: `/v3/projects/a4e0fba6cab34c429c73fcfac7ebfd94/users/eca89f435299415c8448931e73f5b0fc/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.214193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 805/1601] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.215620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bff282-313f-4173-ae08-a9a64102f4e0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.229267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bff282-313f-4173-ae08-a9a64102f4e0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.234747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d732c8a-cc1d-4671-a3f3-2d95d0206d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.235195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a78ba-077a-4c14-85fb-56185f049fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e0fba6cab34c429c73fcfac7ebfd94, user_id=eca89f435299415c8448931e73f5b0fc, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.235352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732c8a-cc1d-4671-a3f3-2d95d0206d2c admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.235632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732c8a-cc1d-4671-a3f3-2d95d0206d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.235917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732c8a-cc1d-4671-a3f3-2d95d0206d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.236821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d732c8a-cc1d-4671-a3f3-2d95d0206d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.238759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d732c8a-cc1d-4671-a3f3-2d95d0206d2c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.238892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a78ba-077a-4c14-85fb-56185f049fb8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.249860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 806/1602] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:28.250716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d732c8a-cc1d-4671-a3f3-2d95d0206d2c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.260466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d732c8a-cc1d-4671-a3f3-2d95d0206d2c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.265193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aca7f-4203-4189-b752-c524832dbe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.265847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aca7f-4203-4189-b752-c524832dbe94 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.266124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aca7f-4203-4189-b752-c524832dbe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.266427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aca7f-4203-4189-b752-c524832dbe94 admin admin] PATH_INFO: `/v3/projects/4c194dd986d442d3bcaa83442a49ff70/users/e9925313d1d446fa81a00cb55348dff7/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.286008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3aca7f-4203-4189-b752-c524832dbe94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c194dd986d442d3bcaa83442a49ff70, user_id=e9925313d1d446fa81a00cb55348dff7, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.291708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3aca7f-4203-4189-b752-c524832dbe94 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.307942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 797/1603] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:28 2022] PUT /identity/v3/projects/a4e0fba6cab34c429c73fcfac7ebfd94/users/eca89f435299415c8448931e73f5b0fc/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:28.309348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06233c-6fa4-4f38-a20b-3b6690ede8f9 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.309636 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06233c-6fa4-4f38-a20b-3b6690ede8f9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.309915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06233c-6fa4-4f38-a20b-3b6690ede8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.328824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ef06233c-6fa4-4f38-a20b-3b6690ede8f9 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:28.329944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ef06233c-6fa4-4f38-a20b-3b6690ede8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.375363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 807/1604] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:28 2022] PUT /identity/v3/projects/4c194dd986d442d3bcaa83442a49ff70/users/e9925313d1d446fa81a00cb55348dff7/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:28.376192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aca7f-4203-4189-b752-c524832dbe94 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.398644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 798/1605] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 8 13:53:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:28.399434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06233c-6fa4-4f38-a20b-3b6690ede8f9 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.410574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06233c-6fa4-4f38-a20b-3b6690ede8f9 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.438873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aca7f-4203-4189-b752-c524832dbe94 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.443519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4490e73-d23a-42ab-8fc1-400f0ff30b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.444176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4490e73-d23a-42ab-8fc1-400f0ff30b22 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.444454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4490e73-d23a-42ab-8fc1-400f0ff30b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.444737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4490e73-d23a-42ab-8fc1-400f0ff30b22 admin admin] PATH_INFO: `/v3/projects/6112c85bb0c2438fbec3d20b53778215/users/ceb18aa17931456ea8e51195f569292c/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.460638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4490e73-d23a-42ab-8fc1-400f0ff30b22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6112c85bb0c2438fbec3d20b53778215, user_id=ceb18aa17931456ea8e51195f569292c, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.463366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4490e73-d23a-42ab-8fc1-400f0ff30b22 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.476640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5ef68-5fb3-4ca0-bf9e-f2e4f0744232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.477257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5ef68-5fb3-4ca0-bf9e-f2e4f0744232 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.477546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5ef68-5fb3-4ca0-bf9e-f2e4f0744232 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.477822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5ef68-5fb3-4ca0-bf9e-f2e4f0744232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.496567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.537238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 808/1606] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:28 2022] PUT /identity/v3/projects/6112c85bb0c2438fbec3d20b53778215/users/ceb18aa17931456ea8e51195f569292c/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:28.538219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4490e73-d23a-42ab-8fc1-400f0ff30b22 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.538979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5ef68-5fb3-4ca0-bf9e-f2e4f0744232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.540699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5ef68-5fb3-4ca0-bf9e-f2e4f0744232 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.552222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 799/1607] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:28.552956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5ef68-5fb3-4ca0-bf9e-f2e4f0744232 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.564030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5ef68-5fb3-4ca0-bf9e-f2e4f0744232 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.610867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4490e73-d23a-42ab-8fc1-400f0ff30b22 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.616721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a562c-40ea-4c4a-b6b2-1171eac32bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.617544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a562c-40ea-4c4a-b6b2-1171eac32bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.617998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a562c-40ea-4c4a-b6b2-1171eac32bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.618325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a562c-40ea-4c4a-b6b2-1171eac32bf3 admin admin] PATH_INFO: `/v3/projects/5775050045e749b1bdb26dddcce9a8ff/users/8294dcc1178045abbcf7a497564eb122/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.622842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fcab1-b84e-480a-bbfc-4098ba37a3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.623684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fcab1-b84e-480a-bbfc-4098ba37a3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.624040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fcab1-b84e-480a-bbfc-4098ba37a3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.624434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fcab1-b84e-480a-bbfc-4098ba37a3f3 admin admin] PATH_INFO: `/v3/projects/2f3bfabfe85447c785cb9cc719688632/users/35e43a4da60d4b55bc353b8895e56b20/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.635956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a562c-40ea-4c4a-b6b2-1171eac32bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5775050045e749b1bdb26dddcce9a8ff, user_id=8294dcc1178045abbcf7a497564eb122, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.639429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a562c-40ea-4c4a-b6b2-1171eac32bf3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.647079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fcab1-b84e-480a-bbfc-4098ba37a3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3bfabfe85447c785cb9cc719688632, user_id=35e43a4da60d4b55bc353b8895e56b20, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.649820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fcab1-b84e-480a-bbfc-4098ba37a3f3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.712165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 809/1608] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:28 2022] PUT /identity/v3/projects/5775050045e749b1bdb26dddcce9a8ff/users/8294dcc1178045abbcf7a497564eb122/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:28.716835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a562c-40ea-4c4a-b6b2-1171eac32bf3 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.734312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.742055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 800/1609] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:28 2022] PUT /identity/v3/projects/2f3bfabfe85447c785cb9cc719688632/users/35e43a4da60d4b55bc353b8895e56b20/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:28.744599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11cf47-0dac-4c1d-92fe-e1e67003d336 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.744599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11cf47-0dac-4c1d-92fe-e1e67003d336 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.745130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11cf47-0dac-4c1d-92fe-e1e67003d336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.766264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6c11cf47-0dac-4c1d-92fe-e1e67003d336 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:28.767496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6c11cf47-0dac-4c1d-92fe-e1e67003d336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.789310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a562c-40ea-4c4a-b6b2-1171eac32bf3 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.832428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 801/1610] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:28.833688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e37b-6cfa-4077-a443-6298e7851d1a None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.833973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e37b-6cfa-4077-a443-6298e7851d1a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.835420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e37b-6cfa-4077-a443-6298e7851d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.855855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878167c-53a3-4867-8c18-e089017c8dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.856623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878167c-53a3-4867-8c18-e089017c8dde admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.856977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878167c-53a3-4867-8c18-e089017c8dde admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.857322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878167c-53a3-4867-8c18-e089017c8dde admin admin] PATH_INFO: `/v3/projects/1346d4105dce4c6e928fcb11cde56854/users/792808dcbf894aa98c671cf43ac21b8b/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.857850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-68e0e37b-6cfa-4077-a443-6298e7851d1a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:28.859141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-68e0e37b-6cfa-4077-a443-6298e7851d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.867069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 802/1611] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:28.867854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0e37b-6cfa-4077-a443-6298e7851d1a None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.877435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878167c-53a3-4867-8c18-e089017c8dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=1346d4105dce4c6e928fcb11cde56854, user_id=792808dcbf894aa98c671cf43ac21b8b, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.880250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878167c-53a3-4867-8c18-e089017c8dde admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.907763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.938115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0e37b-6cfa-4077-a443-6298e7851d1a None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.943612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccb147-21f7-48ba-862f-0b97d86e0328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.944270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb147-21f7-48ba-862f-0b97d86e0328 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.944549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb147-21f7-48ba-862f-0b97d86e0328 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.944824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb147-21f7-48ba-862f-0b97d86e0328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.954388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 810/1612] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:28 2022] PUT /identity/v3/projects/1346d4105dce4c6e928fcb11cde56854/users/792808dcbf894aa98c671cf43ac21b8b/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:28.955946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9ddb-b27b-4edd-a140-aa3e9a1b25f6 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.956229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9ddb-b27b-4edd-a140-aa3e9a1b25f6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.956501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9ddb-b27b-4edd-a140-aa3e9a1b25f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.984042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f77b9ddb-b27b-4edd-a140-aa3e9a1b25f6 None None] MFA Rules not processed for user `ceb18aa17931456ea8e51195f569292c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:28.985256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f77b9ddb-b27b-4edd-a140-aa3e9a1b25f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.003473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccb147-21f7-48ba-862f-0b97d86e0328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.005122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccb147-21f7-48ba-862f-0b97d86e0328 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.015295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 803/1613] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:29.016427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3f8a0-bdab-4b0a-bd9f-ab39406bc43e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.016653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3f8a0-bdab-4b0a-bd9f-ab39406bc43e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.016922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3f8a0-bdab-4b0a-bd9f-ab39406bc43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.038019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6da3f8a0-bdab-4b0a-bd9f-ab39406bc43e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.039277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6da3f8a0-bdab-4b0a-bd9f-ab39406bc43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.055704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 811/1614] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.055704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73977316-5c8b-4a0f-98eb-c529b0137163 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.055704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73977316-5c8b-4a0f-98eb-c529b0137163 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.055704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73977316-5c8b-4a0f-98eb-c529b0137163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.085075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-73977316-5c8b-4a0f-98eb-c529b0137163 None None] MFA Rules not processed for user `35e43a4da60d4b55bc353b8895e56b20`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.086688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-73977316-5c8b-4a0f-98eb-c529b0137163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.113768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 804/1615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.117026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e911b7-8ef3-461a-a25c-a68754b07135 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.117026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e911b7-8ef3-461a-a25c-a68754b07135 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.117026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e911b7-8ef3-461a-a25c-a68754b07135 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.117352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 805/1616] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 8 13:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:29.122986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e911b7-8ef3-461a-a25c-a68754b07135 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.128898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:29.173836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 812/1617] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.173836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73977316-5c8b-4a0f-98eb-c529b0137163 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.183359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e911b7-8ef3-461a-a25c-a68754b07135 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.187495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73977316-5c8b-4a0f-98eb-c529b0137163 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.188069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65437f19-b153-4954-8007-81ffcabecc7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.188765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65437f19-b153-4954-8007-81ffcabecc7d service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.189144 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65437f19-b153-4954-8007-81ffcabecc7d service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.189624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65437f19-b153-4954-8007-81ffcabecc7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.191629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998e7c6-e328-4372-9954-20c22db46b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.192334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998e7c6-e328-4372-9954-20c22db46b52 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.192963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998e7c6-e328-4372-9954-20c22db46b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.193548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998e7c6-e328-4372-9954-20c22db46b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.199821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65437f19-b153-4954-8007-81ffcabecc7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.201445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65437f19-b153-4954-8007-81ffcabecc7d service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.204685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998e7c6-e328-4372-9954-20c22db46b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.206248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998e7c6-e328-4372-9954-20c22db46b52 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.211434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 806/1618] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:29.216173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b632cc7-25ce-453f-a642-3891b403c229 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.216814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b632cc7-25ce-453f-a642-3891b403c229 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.217244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b632cc7-25ce-453f-a642-3891b403c229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.217550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 813/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:29.218488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998e7c6-e328-4372-9954-20c22db46b52 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.229652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998e7c6-e328-4372-9954-20c22db46b52 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.238831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8232594-a0ff-41b9-b2b0-069a49252622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.240786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8232594-a0ff-41b9-b2b0-069a49252622 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.241240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8232594-a0ff-41b9-b2b0-069a49252622 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.241563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8232594-a0ff-41b9-b2b0-069a49252622 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.242309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8232594-a0ff-41b9-b2b0-069a49252622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.243910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2b632cc7-25ce-453f-a642-3891b403c229 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.249265 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8232594-a0ff-41b9-b2b0-069a49252622 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.249397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2b632cc7-25ce-453f-a642-3891b403c229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.255130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 807/1620] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.256879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc961f24-5e36-4a6c-bcc3-8ca444d5a1b5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.257822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc961f24-5e36-4a6c-bcc3-8ca444d5a1b5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.265359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc961f24-5e36-4a6c-bcc3-8ca444d5a1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.285669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-cc961f24-5e36-4a6c-bcc3-8ca444d5a1b5 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.286925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cc961f24-5e36-4a6c-bcc3-8ca444d5a1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.291335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:29.359040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 808/1621] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.359863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc961f24-5e36-4a6c-bcc3-8ca444d5a1b5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.361284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cc961f24-5e36-4a6c-bcc3-8ca444d5a1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.374276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc961f24-5e36-4a6c-bcc3-8ca444d5a1b5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.377733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec2fea-0a71-4d87-9aad-bf79f966499d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.378379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec2fea-0a71-4d87-9aad-bf79f966499d service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.378667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec2fea-0a71-4d87-9aad-bf79f966499d service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.378947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec2fea-0a71-4d87-9aad-bf79f966499d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.388582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec2fea-0a71-4d87-9aad-bf79f966499d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.391099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec2fea-0a71-4d87-9aad-bf79f966499d service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.400127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 809/1622] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:29.401292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec2fea-0a71-4d87-9aad-bf79f966499d service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.427077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 814/1623] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:29.429086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97546c2-bd1b-42cf-a28f-5001679dfafd None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.429487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97546c2-bd1b-42cf-a28f-5001679dfafd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.430170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97546c2-bd1b-42cf-a28f-5001679dfafd None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.431860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 815/1624] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 8 13:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:29.432664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97546c2-bd1b-42cf-a28f-5001679dfafd None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.476470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec2fea-0a71-4d87-9aad-bf79f966499d service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.497442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97546c2-bd1b-42cf-a28f-5001679dfafd None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.501521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e38a71-df96-4e48-aa82-e815b2d67653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.502035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e38a71-df96-4e48-aa82-e815b2d67653 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.505443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e38a71-df96-4e48-aa82-e815b2d67653 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.505443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e38a71-df96-4e48-aa82-e815b2d67653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.537168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675214ec-c48d-4e02-a1f0-71d9449e3759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.537168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675214ec-c48d-4e02-a1f0-71d9449e3759 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.537168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675214ec-c48d-4e02-a1f0-71d9449e3759 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.537807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675214ec-c48d-4e02-a1f0-71d9449e3759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.579059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e38a71-df96-4e48-aa82-e815b2d67653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.581172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e38a71-df96-4e48-aa82-e815b2d67653 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.601244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 816/1625] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:29.602778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e38a71-df96-4e48-aa82-e815b2d67653 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.615115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e38a71-df96-4e48-aa82-e815b2d67653 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.619729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3f759-3092-48a5-9df8-6e2276aa45cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.620791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3f759-3092-48a5-9df8-6e2276aa45cb service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.620926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675214ec-c48d-4e02-a1f0-71d9449e3759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.621596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3f759-3092-48a5-9df8-6e2276aa45cb service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.622074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3f759-3092-48a5-9df8-6e2276aa45cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.622583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675214ec-c48d-4e02-a1f0-71d9449e3759 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.627265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:29.648947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 810/1626] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:29.649932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675214ec-c48d-4e02-a1f0-71d9449e3759 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.673349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675214ec-c48d-4e02-a1f0-71d9449e3759 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.688713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a53dc6-5f42-4632-9283-aab403531b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.689405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a53dc6-5f42-4632-9283-aab403531b6e admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.689705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a53dc6-5f42-4632-9283-aab403531b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.689976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a53dc6-5f42-4632-9283-aab403531b6e admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.690844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a53dc6-5f42-4632-9283-aab403531b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.692711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a53dc6-5f42-4632-9283-aab403531b6e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.698901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:29.743274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3f759-3092-48a5-9df8-6e2276aa45cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.745017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3f759-3092-48a5-9df8-6e2276aa45cb service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.756936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 817/1627] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:29.759262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9506-1263-403e-a834-08900c73e162 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.759570 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9506-1263-403e-a834-08900c73e162 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.760484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9506-1263-403e-a834-08900c73e162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.803874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a53a9506-1263-403e-a834-08900c73e162 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.808700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a53a9506-1263-403e-a834-08900c73e162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.833415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 818/1628] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.859208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 811/1629] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:29.862895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a9506-1263-403e-a834-08900c73e162 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.886568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a9506-1263-403e-a834-08900c73e162 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.906488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8dd95-3d22-4d3f-9f2f-f69db99bdf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.910889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8dd95-3d22-4d3f-9f2f-f69db99bdf70 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.912946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8dd95-3d22-4d3f-9f2f-f69db99bdf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.913331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8dd95-3d22-4d3f-9f2f-f69db99bdf70 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.914301 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8dd95-3d22-4d3f-9f2f-f69db99bdf70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.916755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8dd95-3d22-4d3f-9f2f-f69db99bdf70 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.948553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 819/1630] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:29.954002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a53dc6-5f42-4632-9283-aab403531b6e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.977574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a53dc6-5f42-4632-9283-aab403531b6e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.986826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aec0a9-2510-4aba-803b-46d59c593823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.988254 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aec0a9-2510-4aba-803b-46d59c593823 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.988729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aec0a9-2510-4aba-803b-46d59c593823 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.990397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aec0a9-2510-4aba-803b-46d59c593823 admin admin] PATH_INFO: `/v3/projects/48172adad29240b3a4266cd6657ae257/users/1f87a82b399c433bbd8346b785273e81/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.038366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aec0a9-2510-4aba-803b-46d59c593823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48172adad29240b3a4266cd6657ae257, user_id=1f87a82b399c433bbd8346b785273e81, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:30.041512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aec0a9-2510-4aba-803b-46d59c593823 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:30.203780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 812/1631] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:29 2022] PUT /identity/v3/projects/48172adad29240b3a4266cd6657ae257/users/1f87a82b399c433bbd8346b785273e81/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:30.207676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8dd95-3d22-4d3f-9f2f-f69db99bdf70 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:30.218894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8dd95-3d22-4d3f-9f2f-f69db99bdf70 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:30.318687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7841b-1edd-4775-affb-1185cfabbb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:30.319490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7841b-1edd-4775-affb-1185cfabbb23 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.319842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7841b-1edd-4775-affb-1185cfabbb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.320241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7841b-1edd-4775-affb-1185cfabbb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.320978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7841b-1edd-4775-affb-1185cfabbb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:30.322823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7841b-1edd-4775-affb-1185cfabbb23 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:30.336581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 820/1632] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:30.340749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aec0a9-2510-4aba-803b-46d59c593823 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:30.360124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aec0a9-2510-4aba-803b-46d59c593823 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:30.377323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ee3c0-97d8-4dd2-8e17-575095f5e309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:30.378354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee3c0-97d8-4dd2-8e17-575095f5e309 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.378755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee3c0-97d8-4dd2-8e17-575095f5e309 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.379130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee3c0-97d8-4dd2-8e17-575095f5e309 admin admin] PATH_INFO: `/v3/projects/48172adad29240b3a4266cd6657ae257/users/1f87a82b399c433bbd8346b785273e81/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.383149 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ee3c0-97d8-4dd2-8e17-575095f5e309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48172adad29240b3a4266cd6657ae257, user_id=1f87a82b399c433bbd8346b785273e81, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:30.386213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ee3c0-97d8-4dd2-8e17-575095f5e309 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:30.427668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 813/1633] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:30 2022] PUT /identity/v3/projects/48172adad29240b3a4266cd6657ae257/users/1f87a82b399c433bbd8346b785273e81/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:30.429429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c13a-b532-47ac-9dc3-e74ef48482f3 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.429731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c13a-b532-47ac-9dc3-e74ef48482f3 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.430277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c13a-b532-47ac-9dc3-e74ef48482f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.464505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-2340c13a-b532-47ac-9dc3-e74ef48482f3 None None] MFA Rules not processed for user `1f87a82b399c433bbd8346b785273e81`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:30.465783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2340c13a-b532-47ac-9dc3-e74ef48482f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:30.547887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 821/1634] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:30.912036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ee3c0-97d8-4dd2-8e17-575095f5e309 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:30.986958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ee3c0-97d8-4dd2-8e17-575095f5e309 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:30.992862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccedf3-bd26-4acf-8230-dfb314799100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:30.994480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccedf3-bd26-4acf-8230-dfb314799100 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.994789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccedf3-bd26-4acf-8230-dfb314799100 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.995063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccedf3-bd26-4acf-8230-dfb314799100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.489322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c13a-b532-47ac-9dc3-e74ef48482f3 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.489322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c13a-b532-47ac-9dc3-e74ef48482f3 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.489322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62012e1f-112e-44d6-b177-a5b8eb606c8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.489322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62012e1f-112e-44d6-b177-a5b8eb606c8b service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.502954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62012e1f-112e-44d6-b177-a5b8eb606c8b service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.502954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62012e1f-112e-44d6-b177-a5b8eb606c8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.504031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccedf3-bd26-4acf-8230-dfb314799100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.504031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccedf3-bd26-4acf-8230-dfb314799100 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.504825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62012e1f-112e-44d6-b177-a5b8eb606c8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.504825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62012e1f-112e-44d6-b177-a5b8eb606c8b service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.506016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 814/1635] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:31.506949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 822/1636] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:33.137327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccedf3-bd26-4acf-8230-dfb314799100 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:33.235294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:33.261979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccedf3-bd26-4acf-8230-dfb314799100 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:33.271510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:33.272877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:33.273366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:33.273896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] PATH_INFO: `/v3/users/ceb18aa17931456ea8e51195f569292c` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:33.276252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb18aa17931456ea8e51195f569292c)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:33.280597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: WARNING py.warnings [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:53:33.280597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: warnings.warn(msg) Sep 08 13:53:33.280597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: Sep 08 13:53:33.283921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:33.568652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb18aa17931456ea8e51195f569292c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:33.645583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb18aa17931456ea8e51195f569292c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:33.646616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb18aa17931456ea8e51195f569292c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:33.647507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb18aa17931456ea8e51195f569292c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:33.692547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb18aa17931456ea8e51195f569292c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:33.751989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 815/1637] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:33 2022] DELETE /identity/v3/users/ceb18aa17931456ea8e51195f569292c => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:33.756367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c35343-9610-4ebe-87b5-b2a89a582809 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:33.756847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c35343-9610-4ebe-87b5-b2a89a582809 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:33.756924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c35343-9610-4ebe-87b5-b2a89a582809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:33.811410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-14c35343-9610-4ebe-87b5-b2a89a582809 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:33.812781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-14c35343-9610-4ebe-87b5-b2a89a582809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:33.872001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:34.015976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 823/1638] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:34.037114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:34.038872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:34.216482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11317ebc-a15b-4eab-a3b3-250e6c4acbcc admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:34.228132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3e7c8-e175-48cf-9811-88f56f2bf273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:34.228853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e7c8-e175-48cf-9811-88f56f2bf273 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:34.229218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e7c8-e175-48cf-9811-88f56f2bf273 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:34.229687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e7c8-e175-48cf-9811-88f56f2bf273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:34.259052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3e7c8-e175-48cf-9811-88f56f2bf273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:34.267654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3e7c8-e175-48cf-9811-88f56f2bf273 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:34.300711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 816/1639] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:36.116240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c35343-9610-4ebe-87b5-b2a89a582809 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:36.118792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-14c35343-9610-4ebe-87b5-b2a89a582809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:36.163526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c35343-9610-4ebe-87b5-b2a89a582809 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:36.191651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:36.197718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:36.197718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:36.197718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] PATH_INFO: `/v3/projects/6112c85bb0c2438fbec3d20b53778215` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:36.197718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6112c85bb0c2438fbec3d20b53778215)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:36.197718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: WARNING py.warnings [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:53:36.197718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: warnings.warn(msg) Sep 08 13:53:36.197718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: Sep 08 13:53:36.199111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:36.542326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6112c85bb0c2438fbec3d20b53778215', 'request_id': 'req-75831e32-1e54-4e93-936e-af53c359b3a1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.595344 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6112c85bb0c2438fbec3d20b53778215', 'request_id': 'req-75831e32-1e54-4e93-936e-af53c359b3a1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.668572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6112c85bb0c2438fbec3d20b53778215', 'request_id': 'req-75831e32-1e54-4e93-936e-af53c359b3a1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.687625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] The token cache is being invalidate because project 6112c85bb0c2438fbec3d20b53778215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:36.688375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.689328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.691813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 824/1640] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:36 2022] DELETE /identity/v3/projects/6112c85bb0c2438fbec3d20b53778215 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:36.695923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3e7c8-e175-48cf-9811-88f56f2bf273 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:36.717330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3e7c8-e175-48cf-9811-88f56f2bf273 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:36.841567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:36.842276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:36.842620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:36.842907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] PATH_INFO: `/v3/users/1f87a82b399c433bbd8346b785273e81` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:36.847507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f87a82b399c433bbd8346b785273e81)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:36.849503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:36.860898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:37.111189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f87a82b399c433bbd8346b785273e81'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:37.212779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f87a82b399c433bbd8346b785273e81'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:37.213599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f87a82b399c433bbd8346b785273e81'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:37.220836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f87a82b399c433bbd8346b785273e81'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:37.287044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f87a82b399c433bbd8346b785273e81'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:37.363407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 817/1641] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:36 2022] DELETE /identity/v3/users/1f87a82b399c433bbd8346b785273e81 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:38.793460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:38.795913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:38.807036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:38.810623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:38.834716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75831e32-1e54-4e93-936e-af53c359b3a1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.017552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80078-9675-4b55-8d72-0e1ab9b64d43 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.045014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.046268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.046714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.047142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] PATH_INFO: `/v3/users/bfc74593ee634c619b783f9daa20312c` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.049087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc74593ee634c619b783f9daa20312c)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.051468 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.053519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.056341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.057049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.060054 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] PATH_INFO: `/v3/projects/48172adad29240b3a4266cd6657ae257` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.063148 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48172adad29240b3a4266cd6657ae257)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.067924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.299644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc74593ee634c619b783f9daa20312c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.320159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:39.330430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48172adad29240b3a4266cd6657ae257', 'request_id': 'req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.348713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48172adad29240b3a4266cd6657ae257', 'request_id': 'req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.396251 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc74593ee634c619b783f9daa20312c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.396891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc74593ee634c619b783f9daa20312c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.397522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc74593ee634c619b783f9daa20312c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.411578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a894510e-784b-4b45-b2c9-0f003550d93b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc74593ee634c619b783f9daa20312c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.479179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 818/1642] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:38 2022] DELETE /identity/v3/users/bfc74593ee634c619b783f9daa20312c => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:39.484215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78f73a-c51d-4894-a47c-dc3d647522ef None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.484517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78f73a-c51d-4894-a47c-dc3d647522ef None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.484797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78f73a-c51d-4894-a47c-dc3d647522ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.491614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48172adad29240b3a4266cd6657ae257', 'request_id': 'req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.513213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] The token cache is being invalidate because project 48172adad29240b3a4266cd6657ae257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:39.513514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.519703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-509a5fe1-55ac-4595-a5aa-b0c5ae10c993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.524947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:39.525320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 825/1643] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:38 2022] DELETE /identity/v3/projects/48172adad29240b3a4266cd6657ae257 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:39.535285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447649-95ca-47c0-a497-841bb8596b55 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.535629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447649-95ca-47c0-a497-841bb8596b55 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.535915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447649-95ca-47c0-a497-841bb8596b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.584474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3a78f73a-c51d-4894-a47c-dc3d647522ef None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:39.585825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3a78f73a-c51d-4894-a47c-dc3d647522ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.619246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c1447649-95ca-47c0-a497-841bb8596b55 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:39.620556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c1447649-95ca-47c0-a497-841bb8596b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.736337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 819/1644] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:39.751091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 826/1645] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:39.763976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234db78-6fcc-4687-a566-e25f9ce68478 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.764271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234db78-6fcc-4687-a566-e25f9ce68478 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.764926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234db78-6fcc-4687-a566-e25f9ce68478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.800694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1447649-95ca-47c0-a497-841bb8596b55 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.802525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c1447649-95ca-47c0-a497-841bb8596b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.827808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-8234db78-6fcc-4687-a566-e25f9ce68478 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:39.837165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8234db78-6fcc-4687-a566-e25f9ce68478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.901810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 820/1646] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:39.901810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234db78-6fcc-4687-a566-e25f9ce68478 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.920880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234db78-6fcc-4687-a566-e25f9ce68478 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.936711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.937572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.938694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.940519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.947886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:39.947886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.951129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.961370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1447649-95ca-47c0-a497-841bb8596b55 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.971314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585317d9-c00e-48c9-982c-07bceb140669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.971314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585317d9-c00e-48c9-982c-07bceb140669 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.971314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585317d9-c00e-48c9-982c-07bceb140669 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.971314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585317d9-c00e-48c9-982c-07bceb140669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.996797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585317d9-c00e-48c9-982c-07bceb140669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.003180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585317d9-c00e-48c9-982c-07bceb140669 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.003430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 821/1647] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:40.011248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.030245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 827/1648] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:40.048079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9f391-0250-4ddd-9c04-956ceb8fdf06 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.070774 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133869f-4c70-41ce-a28a-0d78c02a61fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.073959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133869f-4c70-41ce-a28a-0d78c02a61fd admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.080272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133869f-4c70-41ce-a28a-0d78c02a61fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.080782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133869f-4c70-41ce-a28a-0d78c02a61fd admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.082167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133869f-4c70-41ce-a28a-0d78c02a61fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.089291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133869f-4c70-41ce-a28a-0d78c02a61fd admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.205152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 822/1649] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:40.209256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585317d9-c00e-48c9-982c-07bceb140669 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.224644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585317d9-c00e-48c9-982c-07bceb140669 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.394893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076b2c8-6089-434e-af77-b2f3f3a2102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.396120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076b2c8-6089-434e-af77-b2f3f3a2102c admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.396744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076b2c8-6089-434e-af77-b2f3f3a2102c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.397168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076b2c8-6089-434e-af77-b2f3f3a2102c admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.397995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076b2c8-6089-434e-af77-b2f3f3a2102c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.401140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076b2c8-6089-434e-af77-b2f3f3a2102c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.613676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 828/1650] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/users => generated 285 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:40.615019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133869f-4c70-41ce-a28a-0d78c02a61fd admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.632841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133869f-4c70-41ce-a28a-0d78c02a61fd admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.643695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af8d5-444d-4839-9574-b8cfad76ba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.644337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af8d5-444d-4839-9574-b8cfad76ba3a admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.644619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af8d5-444d-4839-9574-b8cfad76ba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.644896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af8d5-444d-4839-9574-b8cfad76ba3a admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.645548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3af8d5-444d-4839-9574-b8cfad76ba3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.647199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3af8d5-444d-4839-9574-b8cfad76ba3a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.677920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 823/1651] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:40.682003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076b2c8-6089-434e-af77-b2f3f3a2102c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.708506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076b2c8-6089-434e-af77-b2f3f3a2102c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.727349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617af118-4dca-4546-8d12-959656078ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.728131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617af118-4dca-4546-8d12-959656078ada admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.729837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617af118-4dca-4546-8d12-959656078ada admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.730268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617af118-4dca-4546-8d12-959656078ada admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.731710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617af118-4dca-4546-8d12-959656078ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.737627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617af118-4dca-4546-8d12-959656078ada admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.776828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 829/1652] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:40.778553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af8d5-444d-4839-9574-b8cfad76ba3a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.797175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af8d5-444d-4839-9574-b8cfad76ba3a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.819813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1668ec-f770-46df-8cd7-4528c98b555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.820508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1668ec-f770-46df-8cd7-4528c98b555e admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.820788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1668ec-f770-46df-8cd7-4528c98b555e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.821061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1668ec-f770-46df-8cd7-4528c98b555e admin admin] PATH_INFO: `/v3/projects/f15dfb8f979045519dd27a1e4254b387/users/40e4e373865d4e31acd71c3aaf35a463/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.861069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1668ec-f770-46df-8cd7-4528c98b555e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15dfb8f979045519dd27a1e4254b387, user_id=40e4e373865d4e31acd71c3aaf35a463, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.872817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1668ec-f770-46df-8cd7-4528c98b555e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.985661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 824/1653] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:40 2022] PUT /identity/v3/projects/f15dfb8f979045519dd27a1e4254b387/users/40e4e373865d4e31acd71c3aaf35a463/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:40.999672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39d34-a5a9-4c55-95ed-cad0398a31cc None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.000169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39d34-a5a9-4c55-95ed-cad0398a31cc None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.000266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39d34-a5a9-4c55-95ed-cad0398a31cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.090990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e1c39d34-a5a9-4c55-95ed-cad0398a31cc None None] MFA Rules not processed for user `40e4e373865d4e31acd71c3aaf35a463`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:41.094768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e1c39d34-a5a9-4c55-95ed-cad0398a31cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:41.165759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:41.310898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 830/1654] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:41.314752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1668ec-f770-46df-8cd7-4528c98b555e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.502053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1668ec-f770-46df-8cd7-4528c98b555e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.519156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b1b15-bf54-49a4-acde-e87440431093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:41.521148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1b15-bf54-49a4-acde-e87440431093 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.522364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1b15-bf54-49a4-acde-e87440431093 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.527868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1b15-bf54-49a4-acde-e87440431093 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.527868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b1b15-bf54-49a4-acde-e87440431093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.530698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b1b15-bf54-49a4-acde-e87440431093 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:41.560125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c39d34-a5a9-4c55-95ed-cad0398a31cc None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.561982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e1c39d34-a5a9-4c55-95ed-cad0398a31cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:41.579409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c39d34-a5a9-4c55-95ed-cad0398a31cc None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.586275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:41.586275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.586275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.586275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] PATH_INFO: `/v3/users/9b91f50cffd149f29d30d68083f749dd` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.590741 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b91f50cffd149f29d30d68083f749dd)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.592851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:41.781420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 825/1655] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:41 2022] POST /identity/v3/projects => generated 369 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:41.788384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b1b15-bf54-49a4-acde-e87440431093 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.839470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:41.926709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b91f50cffd149f29d30d68083f749dd'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:41.955122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b1b15-bf54-49a4-acde-e87440431093 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.963747 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:41.964622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.965143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.965517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.966347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.970521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.028908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b91f50cffd149f29d30d68083f749dd'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.030233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b91f50cffd149f29d30d68083f749dd'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.051956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b91f50cffd149f29d30d68083f749dd'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.075953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 826/1656] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:41 2022] POST /identity/v3/users => generated 291 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:42.076628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-882b416e-b56b-4bd1-9c7d-1254b0c724df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b91f50cffd149f29d30d68083f749dd'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.080217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.082515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:42.118349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 831/1657] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:41 2022] DELETE /identity/v3/users/9b91f50cffd149f29d30d68083f749dd => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:42.124873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac9c3f-bc72-4b31-b408-83a5455c09b1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.125244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac9c3f-bc72-4b31-b408-83a5455c09b1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.125534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac9c3f-bc72-4b31-b408-83a5455c09b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.132328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cd72a-c3fa-4f41-986c-3566d748008e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.186614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-8eac9c3f-bc72-4b31-b408-83a5455c09b1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:42.187810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8eac9c3f-bc72-4b31-b408-83a5455c09b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:42.401495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.402296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.406582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.407944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.408762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.410754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.447545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 827/1658] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:42.449092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.456068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:42.463644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 832/1659] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:42.464677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac9c3f-bc72-4b31-b408-83a5455c09b1 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.487529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac9c3f-bc72-4b31-b408-83a5455c09b1 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.490660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d92db-8276-4852-9218-5cbbf8f76567 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.498058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.499143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.499754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.500045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] PATH_INFO: `/v3/projects/3d4fd87e7e0543eb877d77168438a74d/users/8a61606743f14cf0bc01a0b7a3ee6033/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.502224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.504516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.504949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.505301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] PATH_INFO: `/v3/projects/7538afd39b364087a38b64080f27ec58` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.507737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7538afd39b364087a38b64080f27ec58)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.534466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.539257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4fd87e7e0543eb877d77168438a74d, user_id=8a61606743f14cf0bc01a0b7a3ee6033, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.542093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.614394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 833/1660] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:42 2022] PUT /identity/v3/projects/3d4fd87e7e0543eb877d77168438a74d/users/8a61606743f14cf0bc01a0b7a3ee6033/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:42.615632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.619432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:42.748761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c3cbe-26d3-4dbd-bffb-c411e7147e4a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.749028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7538afd39b364087a38b64080f27ec58', 'request_id': 'req-c9551971-c77a-47fb-8579-97ecb9e21f6f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.779661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7538afd39b364087a38b64080f27ec58', 'request_id': 'req-c9551971-c77a-47fb-8579-97ecb9e21f6f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.868776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7538afd39b364087a38b64080f27ec58', 'request_id': 'req-c9551971-c77a-47fb-8579-97ecb9e21f6f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.869516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:42.899403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e302ff-77f5-4708-aa5f-fe5f90583092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.901116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e302ff-77f5-4708-aa5f-fe5f90583092 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.901389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e302ff-77f5-4708-aa5f-fe5f90583092 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.901837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e302ff-77f5-4708-aa5f-fe5f90583092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.905138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] The token cache is being invalidate because project 7538afd39b364087a38b64080f27ec58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:42.905439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.906206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:42.913487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 828/1661] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:42 2022] DELETE /identity/v3/projects/7538afd39b364087a38b64080f27ec58 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:42.913487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.917765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.122362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:43.131150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e302ff-77f5-4708-aa5f-fe5f90583092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:43.138656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9551971-c77a-47fb-8579-97ecb9e21f6f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:43.142679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e302ff-77f5-4708-aa5f-fe5f90583092 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:43.147754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200dcd5-31f8-41f7-939b-87661c6fbb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:43.148392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200dcd5-31f8-41f7-939b-87661c6fbb3e admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.148672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200dcd5-31f8-41f7-939b-87661c6fbb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.148942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200dcd5-31f8-41f7-939b-87661c6fbb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.149591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200dcd5-31f8-41f7-939b-87661c6fbb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:43.151807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200dcd5-31f8-41f7-939b-87661c6fbb3e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:43.174722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 829/1662] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:43.177277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 834/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:43.187356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2f41-4d09-485d-82c1-f9db03dca236 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.187655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2f41-4d09-485d-82c1-f9db03dca236 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.188103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2f41-4d09-485d-82c1-f9db03dca236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.189555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fb4ee-ccac-48f4-9a19-f804eba4c87d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.189854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fb4ee-ccac-48f4-9a19-f804eba4c87d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.190132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fb4ee-ccac-48f4-9a19-f804eba4c87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.271390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-db5fb4ee-ccac-48f4-9a19-f804eba4c87d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:43.272650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-db5fb4ee-ccac-48f4-9a19-f804eba4c87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.282076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-46ab2f41-4d09-485d-82c1-f9db03dca236 None None] MFA Rules not processed for user `eca89f435299415c8448931e73f5b0fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:43.286187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-46ab2f41-4d09-485d-82c1-f9db03dca236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.310166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 830/1664] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:43.311086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fb4ee-ccac-48f4-9a19-f804eba4c87d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:43.339709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fb4ee-ccac-48f4-9a19-f804eba4c87d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:43.352640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5ce91-9f9a-4f19-a0e2-f99f1cc301f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:43.353338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ce91-9f9a-4f19-a0e2-f99f1cc301f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.353619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ce91-9f9a-4f19-a0e2-f99f1cc301f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.356258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ce91-9f9a-4f19-a0e2-f99f1cc301f0 admin admin] PATH_INFO: `/v3/projects/3d4fd87e7e0543eb877d77168438a74d/users/8a61606743f14cf0bc01a0b7a3ee6033/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.362610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5ce91-9f9a-4f19-a0e2-f99f1cc301f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4fd87e7e0543eb877d77168438a74d, user_id=8a61606743f14cf0bc01a0b7a3ee6033, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:43.370118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5ce91-9f9a-4f19-a0e2-f99f1cc301f0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:43.401420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 831/1665] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:43 2022] PUT /identity/v3/projects/3d4fd87e7e0543eb877d77168438a74d/users/8a61606743f14cf0bc01a0b7a3ee6033/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:43.414051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96aee07-b3e3-48a6-85c4-a291592558c6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.415468 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96aee07-b3e3-48a6-85c4-a291592558c6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.415761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96aee07-b3e3-48a6-85c4-a291592558c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.469330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-a96aee07-b3e3-48a6-85c4-a291592558c6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:43.470614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a96aee07-b3e3-48a6-85c4-a291592558c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.511927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 835/1666] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:43.516879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ad2cb-b32f-4c49-a5bf-f82834694c5d None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.517183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ad2cb-b32f-4c49-a5bf-f82834694c5d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.517467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ad2cb-b32f-4c49-a5bf-f82834694c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.614217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-179ad2cb-b32f-4c49-a5bf-f82834694c5d None None] MFA Rules not processed for user `8a61606743f14cf0bc01a0b7a3ee6033`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:43.615654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-179ad2cb-b32f-4c49-a5bf-f82834694c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.696779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 832/1667] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:43.697743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96aee07-b3e3-48a6-85c4-a291592558c6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:43.703455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a96aee07-b3e3-48a6-85c4-a291592558c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.787006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 836/1668] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:43.796568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ad2cb-b32f-4c49-a5bf-f82834694c5d None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:43.821915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ad2cb-b32f-4c49-a5bf-f82834694c5d None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:43.835556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:43.836295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.836727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.837108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.837879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:43.838341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:43.842722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:43.876412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 837/1669] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:43.878999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:43.883894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.888946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96aee07-b3e3-48a6-85c4-a291592558c6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:43.944890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5da8a-327d-423c-827e-8228a31bf6e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:43.945538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5da8a-327d-423c-827e-8228a31bf6e3 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.945836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5da8a-327d-423c-827e-8228a31bf6e3 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.946108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5da8a-327d-423c-827e-8228a31bf6e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:44.126240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa376e0-f741-4976-a879-2ca04101ac49 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:44.140771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc105-7b2e-4387-b0f5-c7ffbcae5558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:44.142538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc105-7b2e-4387-b0f5-c7ffbcae5558 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:44.143128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc105-7b2e-4387-b0f5-c7ffbcae5558 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:44.144857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc105-7b2e-4387-b0f5-c7ffbcae5558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:44.189912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fc105-7b2e-4387-b0f5-c7ffbcae5558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:44.191526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fc105-7b2e-4387-b0f5-c7ffbcae5558 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:44.198712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5da8a-327d-423c-827e-8228a31bf6e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:44.201556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5da8a-327d-423c-827e-8228a31bf6e3 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:44.214400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 838/1670] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:44.229178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc105-7b2e-4387-b0f5-c7ffbcae5558 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:44.255825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 833/1671] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:44.260171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ab2d-a947-48a8-ad5b-0c4e94b24fd9 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:44.273054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ab2d-a947-48a8-ad5b-0c4e94b24fd9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:44.275156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ab2d-a947-48a8-ad5b-0c4e94b24fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:44.314467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc105-7b2e-4387-b0f5-c7ffbcae5558 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:44.355048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d04d1a-fbbf-415d-a261-d16241143caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:44.356205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d04d1a-fbbf-415d-a261-d16241143caf admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:44.359362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d04d1a-fbbf-415d-a261-d16241143caf admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:44.359928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d04d1a-fbbf-415d-a261-d16241143caf admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:44.364502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d04d1a-fbbf-415d-a261-d16241143caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:44.367583 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d04d1a-fbbf-415d-a261-d16241143caf admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:44.436601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-94a9ab2d-a947-48a8-ad5b-0c4e94b24fd9 None None] MFA Rules not processed for user `8294dcc1178045abbcf7a497564eb122`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:44.440156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-94a9ab2d-a947-48a8-ad5b-0c4e94b24fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:44.472839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:44.558833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 839/1672] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:44 2022] POST /identity/v3/projects => generated 357 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:44.558833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d04d1a-fbbf-415d-a261-d16241143caf admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:44.687156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:44.730791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d04d1a-fbbf-415d-a261-d16241143caf admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:44.752156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3065c4-0c2e-4a9e-9ba6-9aa4bf57b231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:44.753172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3065c4-0c2e-4a9e-9ba6-9aa4bf57b231 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:44.755828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3065c4-0c2e-4a9e-9ba6-9aa4bf57b231 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:44.755828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3065c4-0c2e-4a9e-9ba6-9aa4bf57b231 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:44.755828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3065c4-0c2e-4a9e-9ba6-9aa4bf57b231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:44.763341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3065c4-0c2e-4a9e-9ba6-9aa4bf57b231 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:44.782916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 834/1673] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 524 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:44.796324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad744-40af-4501-becd-3a6f66ee199b None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:44.797156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad744-40af-4501-becd-3a6f66ee199b None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:44.797445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad744-40af-4501-becd-3a6f66ee199b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:44.846802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3a9ad744-40af-4501-becd-3a6f66ee199b None None] MFA Rules not processed for user `8294dcc1178045abbcf7a497564eb122`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:44.848740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3a9ad744-40af-4501-becd-3a6f66ee199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:44.954216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 840/1674] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:44 2022] POST /identity/v3/users => generated 279 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:44.956833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3065c4-0c2e-4a9e-9ba6-9aa4bf57b231 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:44.991151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3065c4-0c2e-4a9e-9ba6-9aa4bf57b231 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:45.006390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 835/1675] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:45.006682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db52a35-7af7-4d3c-b228-cc56858929cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:45.006682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52a35-7af7-4d3c-b228-cc56858929cf admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.006682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52a35-7af7-4d3c-b228-cc56858929cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.006682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52a35-7af7-4d3c-b228-cc56858929cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.007268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db52a35-7af7-4d3c-b228-cc56858929cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:45.011690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db52a35-7af7-4d3c-b228-cc56858929cf admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:45.021046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ad744-40af-4501-becd-3a6f66ee199b None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:45.022767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3a9ad744-40af-4501-becd-3a6f66ee199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:45.043797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 841/1676] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:45.053098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db52a35-7af7-4d3c-b228-cc56858929cf admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:45.098474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db52a35-7af7-4d3c-b228-cc56858929cf admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:45.124072 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612339d-0bd9-4b50-905e-9c14ee35a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:45.127350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612339d-0bd9-4b50-905e-9c14ee35a947 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.129127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612339d-0bd9-4b50-905e-9c14ee35a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.130099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612339d-0bd9-4b50-905e-9c14ee35a947 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.131780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612339d-0bd9-4b50-905e-9c14ee35a947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:45.140213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612339d-0bd9-4b50-905e-9c14ee35a947 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:45.167678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 842/1677] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:45.172997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612339d-0bd9-4b50-905e-9c14ee35a947 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:45.198254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612339d-0bd9-4b50-905e-9c14ee35a947 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:45.206520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9227ab-57db-4dfa-b21a-2af390264fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:45.207681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9227ab-57db-4dfa-b21a-2af390264fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.208099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9227ab-57db-4dfa-b21a-2af390264fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.208535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9227ab-57db-4dfa-b21a-2af390264fdc admin admin] PATH_INFO: `/v3/projects/9f46ec1cc9144fbfa5f3db291668ca67/users/63228e006c2047208959b5a27d0a2aae/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.232188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9227ab-57db-4dfa-b21a-2af390264fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f46ec1cc9144fbfa5f3db291668ca67, user_id=63228e006c2047208959b5a27d0a2aae, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:45.236312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9227ab-57db-4dfa-b21a-2af390264fdc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:45.275492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ad744-40af-4501-becd-3a6f66ee199b None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:45.289289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:45.290349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.290761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.291178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.303959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:45.305872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:45.332737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 836/1678] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:45.333880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:45.357535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:45.365307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 843/1679] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:45 2022] PUT /identity/v3/projects/9f46ec1cc9144fbfa5f3db291668ca67/users/63228e006c2047208959b5a27d0a2aae/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:45.378710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaeb3b-71cf-4737-9f76-82a318c710fa None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.379228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaeb3b-71cf-4737-9f76-82a318c710fa None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.379798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaeb3b-71cf-4737-9f76-82a318c710fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.402257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dcbdb-397b-41a2-b576-7b366f94a85a service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:45.451625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-cabaeb3b-71cf-4737-9f76-82a318c710fa None None] MFA Rules not processed for user `e9925313d1d446fa81a00cb55348dff7`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:45.452916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cabaeb3b-71cf-4737-9f76-82a318c710fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:45.608250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:45.608250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.608250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.608250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] PATH_INFO: `/v3/projects/7711a822e7ef4fbda3d19219739b228e` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.608804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7711a822e7ef4fbda3d19219739b228e)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:45.615056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:45.620971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 844/1680] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:45.624797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f04b6-ea4e-44f7-8d25-91f75a4efcf5 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.626506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f04b6-ea4e-44f7-8d25-91f75a4efcf5 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.626911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f04b6-ea4e-44f7-8d25-91f75a4efcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.715075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-539f04b6-ea4e-44f7-8d25-91f75a4efcf5 None None] MFA Rules not processed for user `63228e006c2047208959b5a27d0a2aae`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:45.720257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-539f04b6-ea4e-44f7-8d25-91f75a4efcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:45.810442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7711a822e7ef4fbda3d19219739b228e', 'request_id': 'req-69e44d73-509b-4146-9de6-17eb9a8ac8c2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:45.827431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7711a822e7ef4fbda3d19219739b228e', 'request_id': 'req-69e44d73-509b-4146-9de6-17eb9a8ac8c2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:45.870891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 845/1681] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:45.873310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a562-dd2d-433f-a828-f2fa5c0c46c3 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.873708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a562-dd2d-433f-a828-f2fa5c0c46c3 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.874157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a562-dd2d-433f-a828-f2fa5c0c46c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.886880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:45.938536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7711a822e7ef4fbda3d19219739b228e', 'request_id': 'req-69e44d73-509b-4146-9de6-17eb9a8ac8c2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:45.961451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] The token cache is being invalidate because project 7711a822e7ef4fbda3d19219739b228e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:45.963148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:45.965818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d5f8a562-dd2d-433f-a828-f2fa5c0c46c3 None None] MFA Rules not processed for user `792808dcbf894aa98c671cf43ac21b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:45.966422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:45.968192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d5f8a562-dd2d-433f-a828-f2fa5c0c46c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:45.974241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 837/1682] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:45 2022] DELETE /identity/v3/projects/7711a822e7ef4fbda3d19219739b228e => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:45.974241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:45.976521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.130171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e44d73-509b-4146-9de6-17eb9a8ac8c2 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:46.139163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f6f6e-70b1-4efa-9fa5-ad5455dfd0ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:46.139699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6f6e-70b1-4efa-9fa5-ad5455dfd0ce service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.140090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6f6e-70b1-4efa-9fa5-ad5455dfd0ce service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.140496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6f6e-70b1-4efa-9fa5-ad5455dfd0ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.143616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-af5f6f6e-70b1-4efa-9fa5-ad5455dfd0ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.169902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 846/1683] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:46.170925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8a562-dd2d-433f-a828-f2fa5c0c46c3 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:46.192815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d5f8a562-dd2d-433f-a828-f2fa5c0c46c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.257568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f6f6e-70b1-4efa-9fa5-ad5455dfd0ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:46.259589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f6f6e-70b1-4efa-9fa5-ad5455dfd0ce service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:46.294527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 838/1684] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:46.294527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1a329-558f-4f7f-bcf9-5fadffbb4703 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.294527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1a329-558f-4f7f-bcf9-5fadffbb4703 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.294527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1a329-558f-4f7f-bcf9-5fadffbb4703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.320677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:46.342646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8a562-dd2d-433f-a828-f2fa5c0c46c3 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:46.362266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba04d6-acf3-48a9-97df-fa0fbad5adcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:46.362987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba04d6-acf3-48a9-97df-fa0fbad5adcb admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.363786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba04d6-acf3-48a9-97df-fa0fbad5adcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.365045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba04d6-acf3-48a9-97df-fa0fbad5adcb admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.366537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-8af1a329-558f-4f7f-bcf9-5fadffbb4703 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:46.366861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba04d6-acf3-48a9-97df-fa0fbad5adcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:46.370328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8af1a329-558f-4f7f-bcf9-5fadffbb4703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.372014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba04d6-acf3-48a9-97df-fa0fbad5adcb admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:46.394585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 839/1685] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:46.396150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8a57f-9659-44ea-b693-bb335e2000f6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.396480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8a57f-9659-44ea-b693-bb335e2000f6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.396807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8a57f-9659-44ea-b693-bb335e2000f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.427796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-add8a57f-9659-44ea-b693-bb335e2000f6 None None] MFA Rules not processed for user `792808dcbf894aa98c671cf43ac21b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:46.429520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-add8a57f-9659-44ea-b693-bb335e2000f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.441738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 840/1686] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:46.441738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc1174-732b-4e4c-820c-542efd2bea8d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.441738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc1174-732b-4e4c-820c-542efd2bea8d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.441738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc1174-732b-4e4c-820c-542efd2bea8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.470207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c8fc1174-732b-4e4c-820c-542efd2bea8d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:46.471860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c8fc1174-732b-4e4c-820c-542efd2bea8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.490014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 841/1687] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:46.492094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342468b4-bd8e-466e-ac61-f836c600769b None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.492200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342468b4-bd8e-466e-ac61-f836c600769b None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.492482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342468b4-bd8e-466e-ac61-f836c600769b None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.494827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 842/1688] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 8 13:53:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:46.495852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342468b4-bd8e-466e-ac61-f836c600769b None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:46.517717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342468b4-bd8e-466e-ac61-f836c600769b None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:46.579444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:46.580332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.580666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.580941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.581815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:46.582129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:46.583763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:46.591202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 847/1689] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:46 2022] POST /identity/v3/projects => generated 357 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:46.595075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f302564-6caf-4f90-bee0-059266c5fc7b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.595749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f302564-6caf-4f90-bee0-059266c5fc7b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.596420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f302564-6caf-4f90-bee0-059266c5fc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.605830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 843/1690] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:46.606989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:46.673417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e107-36b4-4f4b-98b9-d1b27b022526 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:46.712305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0f302564-6caf-4f90-bee0-059266c5fc7b None None] MFA Rules not processed for user `a71d353c1bd849028930261e9306fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:46.719780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0f302564-6caf-4f90-bee0-059266c5fc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.894652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 848/1691] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:46.894652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f302564-6caf-4f90-bee0-059266c5fc7b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:46.894891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa025e-1da5-4f11-91cb-1312d293da5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:46.894891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa025e-1da5-4f11-91cb-1312d293da5a service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.894891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa025e-1da5-4f11-91cb-1312d293da5a service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.894891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa025e-1da5-4f11-91cb-1312d293da5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.981858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f302564-6caf-4f90-bee0-059266c5fc7b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:46.983406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa025e-1da5-4f11-91cb-1312d293da5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:46.988262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:46.989354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.989643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.989923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.990699 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:46.990811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa025e-1da5-4f11-91cb-1312d293da5a service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:46.992651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:47.042469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 844/1692] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:47.042469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa025e-1da5-4f11-91cb-1312d293da5a service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:47.042469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa025e-1da5-4f11-91cb-1312d293da5a service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:47.042469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:47.044308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:47.044308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:47.044308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:47.044308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:47.054313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:47.151470 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 849/1693] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:46 2022] POST /identity/v3/users => generated 285 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:47.152505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:47.156725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:47.189028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b221e52-7f4a-49a3-977a-a7753af98c5a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:47.199024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2cf3b-0877-4554-bb70-d98f15462742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:47.199024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2cf3b-0877-4554-bb70-d98f15462742 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:47.199024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2cf3b-0877-4554-bb70-d98f15462742 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:47.199024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2cf3b-0877-4554-bb70-d98f15462742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:47.243668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2cf3b-0877-4554-bb70-d98f15462742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:47.245564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 845/1694] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:47 2022] POST /identity/v3/projects => generated 371 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:47.248003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:47.248328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2cf3b-0877-4554-bb70-d98f15462742 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:47.253042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:47.269129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 850/1695] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:47.271026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2cf3b-0877-4554-bb70-d98f15462742 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:47.308355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2cf3b-0877-4554-bb70-d98f15462742 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:47.527307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913c74d-8fd4-4ab0-9216-edefe1ac0f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:47.528193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913c74d-8fd4-4ab0-9216-edefe1ac0f81 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:47.528625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913c74d-8fd4-4ab0-9216-edefe1ac0f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:47.529906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913c74d-8fd4-4ab0-9216-edefe1ac0f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:47.531075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913c74d-8fd4-4ab0-9216-edefe1ac0f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:47.533336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913c74d-8fd4-4ab0-9216-edefe1ac0f81 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:47.546164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:47.568328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 851/1696] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:47.570011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913c74d-8fd4-4ab0-9216-edefe1ac0f81 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:47.586270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913c74d-8fd4-4ab0-9216-edefe1ac0f81 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:47.600328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fa02a-b65f-41f0-a830-8e4b163f3c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:47.601112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fa02a-b65f-41f0-a830-8e4b163f3c21 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:47.601249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fa02a-b65f-41f0-a830-8e4b163f3c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:47.601528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fa02a-b65f-41f0-a830-8e4b163f3c21 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:47.602193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fa02a-b65f-41f0-a830-8e4b163f3c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:47.630594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32e95c-455e-4c31-ba75-9fd995b9c3b3 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:47.635356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fa02a-b65f-41f0-a830-8e4b163f3c21 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:47.646634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:47.647417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:47.648854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:47.648854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] PATH_INFO: `/v3/users/35e43a4da60d4b55bc353b8895e56b20` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:47.650883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e43a4da60d4b55bc353b8895e56b20)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:47.662165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:47.799216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 852/1697] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:47 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:47.799216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fa02a-b65f-41f0-a830-8e4b163f3c21 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:47.820458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fa02a-b65f-41f0-a830-8e4b163f3c21 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:47.836260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e761d5-a3b7-4796-ba34-8ba151f687c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:47.836968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e761d5-a3b7-4796-ba34-8ba151f687c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:47.837256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e761d5-a3b7-4796-ba34-8ba151f687c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:47.837537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e761d5-a3b7-4796-ba34-8ba151f687c6 admin admin] PATH_INFO: `/v3/projects/744c89f7a00942bf8df77a7670af2046/users/42db8520a6974a91a9c132217be3af7d/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:47.870504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e761d5-a3b7-4796-ba34-8ba151f687c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744c89f7a00942bf8df77a7670af2046, user_id=42db8520a6974a91a9c132217be3af7d, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:47.877379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e761d5-a3b7-4796-ba34-8ba151f687c6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:47.898904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:47.964200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e43a4da60d4b55bc353b8895e56b20'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:47.975078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 853/1698] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:47 2022] PUT /identity/v3/projects/744c89f7a00942bf8df77a7670af2046/users/42db8520a6974a91a9c132217be3af7d/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:47.976074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e761d5-a3b7-4796-ba34-8ba151f687c6 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:48.096132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e43a4da60d4b55bc353b8895e56b20'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:48.099539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e43a4da60d4b55bc353b8895e56b20'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:48.099539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e43a4da60d4b55bc353b8895e56b20'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:48.116390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b74e4a-13e2-4b23-aaa5-a113c52c937d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e43a4da60d4b55bc353b8895e56b20'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:48.128339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e761d5-a3b7-4796-ba34-8ba151f687c6 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:48.133471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:48.133636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.133862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.134156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:48.134832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:48.137412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:48.161927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 854/1699] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:48.163252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:48.171788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:48.196674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867bd48-78a8-46b6-9ee6-0651c440ce9e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:48.210989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22542cf8-4890-427c-9611-e846c2e980c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:48.211625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22542cf8-4890-427c-9611-e846c2e980c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.211911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22542cf8-4890-427c-9611-e846c2e980c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.212194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22542cf8-4890-427c-9611-e846c2e980c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:48.212853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22542cf8-4890-427c-9611-e846c2e980c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:48.215646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22542cf8-4890-427c-9611-e846c2e980c2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:48.242266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 846/1700] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:47 2022] DELETE /identity/v3/users/35e43a4da60d4b55bc353b8895e56b20 => generated 0 bytes in 996 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:48.247792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b026d-fcb7-4346-a4d7-73480020dfee None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.248240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b026d-fcb7-4346-a4d7-73480020dfee None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.249276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b026d-fcb7-4346-a4d7-73480020dfee None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:48.256010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 847/1701] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 8 13:53:48 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:48.256978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b026d-fcb7-4346-a4d7-73480020dfee None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:48.262490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ef3b026d-fcb7-4346-a4d7-73480020dfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:48.281721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 855/1702] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:48.286995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7a5da-2bbe-4906-a280-2997e11518b9 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.288028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7a5da-2bbe-4906-a280-2997e11518b9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.289035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7a5da-2bbe-4906-a280-2997e11518b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:48.346171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-88b7a5da-2bbe-4906-a280-2997e11518b9 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:48.363482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-88b7a5da-2bbe-4906-a280-2997e11518b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:48.469065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b026d-fcb7-4346-a4d7-73480020dfee None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:48.475937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc4b34-9fb3-43e9-9a27-392d41e022ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:48.476720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc4b34-9fb3-43e9-9a27-392d41e022ce admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.477001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc4b34-9fb3-43e9-9a27-392d41e022ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.478772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc4b34-9fb3-43e9-9a27-392d41e022ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:48.479843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc4b34-9fb3-43e9-9a27-392d41e022ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:48.488678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc4b34-9fb3-43e9-9a27-392d41e022ce admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:48.513676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 848/1703] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:48.518234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f166059-548c-4b26-b7aa-62ea439ff3f0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.518535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f166059-548c-4b26-b7aa-62ea439ff3f0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.519278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f166059-548c-4b26-b7aa-62ea439ff3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:48.572980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-8f166059-548c-4b26-b7aa-62ea439ff3f0 None None] MFA Rules not processed for user `a71d353c1bd849028930261e9306fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:48.580884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8f166059-548c-4b26-b7aa-62ea439ff3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:48.586156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 856/1704] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:48.587254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7a5da-2bbe-4906-a280-2997e11518b9 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:48.624053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7a5da-2bbe-4906-a280-2997e11518b9 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:48.649173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48cf7e-1a40-4e05-aa3e-89b1cec6a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:48.650527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48cf7e-1a40-4e05-aa3e-89b1cec6a59f admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.651334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48cf7e-1a40-4e05-aa3e-89b1cec6a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.653882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48cf7e-1a40-4e05-aa3e-89b1cec6a59f admin admin] PATH_INFO: `/v3/projects/744c89f7a00942bf8df77a7670af2046/users/42db8520a6974a91a9c132217be3af7d/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:48.660506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48cf7e-1a40-4e05-aa3e-89b1cec6a59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=744c89f7a00942bf8df77a7670af2046, user_id=42db8520a6974a91a9c132217be3af7d, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:48.663574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48cf7e-1a40-4e05-aa3e-89b1cec6a59f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:48.708505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 857/1705] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:48 2022] PUT /identity/v3/projects/744c89f7a00942bf8df77a7670af2046/users/42db8520a6974a91a9c132217be3af7d/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:48.711626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48cf7e-1a40-4e05-aa3e-89b1cec6a59f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:48.796640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 849/1706] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:53:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:48.799112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f166059-548c-4b26-b7aa-62ea439ff3f0 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:48.803073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8f166059-548c-4b26-b7aa-62ea439ff3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:48.826528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f166059-548c-4b26-b7aa-62ea439ff3f0 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:48.951615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8beffe5-43b2-4f41-8941-c7b2bc78f2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:48.952180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beffe5-43b2-4f41-8941-c7b2bc78f2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.952463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beffe5-43b2-4f41-8941-c7b2bc78f2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.952739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beffe5-43b2-4f41-8941-c7b2bc78f2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:48.965852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48cf7e-1a40-4e05-aa3e-89b1cec6a59f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:48.979363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:48.980651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:48.981479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:48.983807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] PATH_INFO: `/v3/projects/708847196d6440d5886ef0f6d77e7ddc/users/3f3054d09a3441509b26cf53714214b6/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.002663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=708847196d6440d5886ef0f6d77e7ddc, user_id=3f3054d09a3441509b26cf53714214b6, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:49.005985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:49.008076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8beffe5-43b2-4f41-8941-c7b2bc78f2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:49.009533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8beffe5-43b2-4f41-8941-c7b2bc78f2f2 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:49.038042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 850/1707] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:49.039879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d3f2-a762-4944-8163-de7ffae0cfb1 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.040178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d3f2-a762-4944-8163-de7ffae0cfb1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.040459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d3f2-a762-4944-8163-de7ffae0cfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.099962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e219d3f2-a762-4944-8163-de7ffae0cfb1 None None] MFA Rules not processed for user `42db8520a6974a91a9c132217be3af7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:49.101998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e219d3f2-a762-4944-8163-de7ffae0cfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.110012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 858/1708] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:48 2022] PUT /identity/v3/projects/708847196d6440d5886ef0f6d77e7ddc/users/3f3054d09a3441509b26cf53714214b6/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:49.114261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:49.123037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.253027 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 851/1709] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:49.254459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d3f2-a762-4944-8163-de7ffae0cfb1 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:49.270210 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e219d3f2-a762-4944-8163-de7ffae0cfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.306260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bc70a-3d7b-4b0d-9cbb-9d6afebf15a2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:49.330188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f71c51-cc08-45eb-8392-35baec6ed4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:49.331075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f71c51-cc08-45eb-8392-35baec6ed4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.331458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f71c51-cc08-45eb-8392-35baec6ed4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.331832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f71c51-cc08-45eb-8392-35baec6ed4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.367731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f71c51-cc08-45eb-8392-35baec6ed4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:49.371084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f71c51-cc08-45eb-8392-35baec6ed4f8 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:49.400063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 859/1710] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:49.402377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171dc22-9a7a-49e8-ac6c-060fcf250137 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.403177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171dc22-9a7a-49e8-ac6c-060fcf250137 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.403463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171dc22-9a7a-49e8-ac6c-060fcf250137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.455980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a171dc22-9a7a-49e8-ac6c-060fcf250137 None None] MFA Rules not processed for user `3f3054d09a3441509b26cf53714214b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:49.458417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a171dc22-9a7a-49e8-ac6c-060fcf250137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.460737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d3f2-a762-4944-8163-de7ffae0cfb1 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:49.474053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:49.480112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.480403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.481571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.490444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.516813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:49.520818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:49.540356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 852/1711] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:49.541174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:49.545141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.558867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 860/1712] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:49.561819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171dc22-9a7a-49e8-ac6c-060fcf250137 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:49.563531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a171dc22-9a7a-49e8-ac6c-060fcf250137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.674281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdd589-e473-4d14-ba22-86f2ae5ff9e0 service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:49.685582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fec06-cb89-460a-bc40-e7c438154833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:49.686243 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fec06-cb89-460a-bc40-e7c438154833 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.686787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fec06-cb89-460a-bc40-e7c438154833 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.687068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fec06-cb89-460a-bc40-e7c438154833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.695071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171dc22-9a7a-49e8-ac6c-060fcf250137 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:49.701310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:49.702865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.702865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.702865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] PATH_INFO: `/v3/users/40e4e373865d4e31acd71c3aaf35a463` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.704265 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e4e373865d4e31acd71c3aaf35a463)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:49.706942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:49.709828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fec06-cb89-460a-bc40-e7c438154833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:49.711342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fec06-cb89-460a-bc40-e7c438154833 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:49.714595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:49.744501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 853/1713] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:49 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:49.764639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fec06-cb89-460a-bc40-e7c438154833 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:49.796453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fec06-cb89-460a-bc40-e7c438154833 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:49.821141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad89dd1-d28b-462a-9ffb-3157744d2d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:49.821980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89dd1-d28b-462a-9ffb-3157744d2d68 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.822313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89dd1-d28b-462a-9ffb-3157744d2d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.822584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89dd1-d28b-462a-9ffb-3157744d2d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.826416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad89dd1-d28b-462a-9ffb-3157744d2d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:49.829685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad89dd1-d28b-462a-9ffb-3157744d2d68 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:49.959410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 854/1714] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:49 2022] POST /identity/v3/projects => generated 371 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:49.963970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad89dd1-d28b-462a-9ffb-3157744d2d68 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:49.972558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e4e373865d4e31acd71c3aaf35a463'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:50.080399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e4e373865d4e31acd71c3aaf35a463'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:50.081191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e4e373865d4e31acd71c3aaf35a463'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:50.144482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e4e373865d4e31acd71c3aaf35a463'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:50.168445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e4e373865d4e31acd71c3aaf35a463'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:50.184015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 861/1715] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:49 2022] DELETE /identity/v3/users/40e4e373865d4e31acd71c3aaf35a463 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:50.188516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:50.192847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.203493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad89dd1-d28b-462a-9ffb-3157744d2d68 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:50.325973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a707d3-853e-4e7c-97c6-9fb9708b5422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:50.326648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a707d3-853e-4e7c-97c6-9fb9708b5422 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.326999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a707d3-853e-4e7c-97c6-9fb9708b5422 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.327296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a707d3-853e-4e7c-97c6-9fb9708b5422 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.328378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a707d3-853e-4e7c-97c6-9fb9708b5422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:50.331798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a707d3-853e-4e7c-97c6-9fb9708b5422 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:50.354519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1db246-b926-4afa-9275-e97847ad19fa admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:50.355900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:50.370999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:50.371820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.372107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.372472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.374437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.426007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 855/1716] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:49 2022] POST /identity/v3/users => generated 292 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:50.427844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4339d-4189-4f2f-9260-e3bcdeace15c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.428198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4339d-4189-4f2f-9260-e3bcdeace15c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.429475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4339d-4189-4f2f-9260-e3bcdeace15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.495219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-39a4339d-4189-4f2f-9260-e3bcdeace15c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:50.500094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-39a4339d-4189-4f2f-9260-e3bcdeace15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.519642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 856/1717] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:50.521730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4339d-4189-4f2f-9260-e3bcdeace15c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:50.526418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-39a4339d-4189-4f2f-9260-e3bcdeace15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.549451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4339d-4189-4f2f-9260-e3bcdeace15c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:50.557496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:50.558513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.560197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.560710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.561865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:50.566199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:50.583562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 857/1718] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:50.584725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:50.588158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.647687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8717c-e432-4549-af85-fdf2b4ae2078 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:50.647687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:50.647687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.647687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.647965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] PATH_INFO: `/v3/projects/2f3bfabfe85447c785cb9cc719688632` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.647965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3bfabfe85447c785cb9cc719688632)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:50.647965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:50.648138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:50.648138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:50.656646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 862/1719] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:50.657668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:50.670190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49551-80fa-41c6-be8a-83721ef84ac1 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:50.684609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:50.684609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.684609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.684609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] PATH_INFO: `/v3/projects/4086e880f09a4f71a3a44efa4a5611bb/users/259683c782a1439ab402bdc3bf7caed8/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.737258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4086e880f09a4f71a3a44efa4a5611bb, user_id=259683c782a1439ab402bdc3bf7caed8, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:50.745572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:50.860926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 863/1720] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:50 2022] PUT /identity/v3/projects/4086e880f09a4f71a3a44efa4a5611bb/users/259683c782a1439ab402bdc3bf7caed8/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:50.930175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:50.930175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.931772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3bfabfe85447c785cb9cc719688632', 'request_id': 'req-9790570b-84a0-466f-9ecd-48e7fa22d0fa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:50.931772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3bfabfe85447c785cb9cc719688632', 'request_id': 'req-9790570b-84a0-466f-9ecd-48e7fa22d0fa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:50.981524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec44bf-10a1-40fe-9260-354854c19727 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:50.993329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939da77-d281-4f3a-aa36-a022bdb13ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:50.993329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939da77-d281-4f3a-aa36-a022bdb13ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.993329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939da77-d281-4f3a-aa36-a022bdb13ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.993329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939da77-d281-4f3a-aa36-a022bdb13ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.135433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3bfabfe85447c785cb9cc719688632', 'request_id': 'req-9790570b-84a0-466f-9ecd-48e7fa22d0fa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:51.135571 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939da77-d281-4f3a-aa36-a022bdb13ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:51.141720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939da77-d281-4f3a-aa36-a022bdb13ce4 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:51.151444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] The token cache is being invalidate because project 2f3bfabfe85447c785cb9cc719688632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:51.157322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:51.164610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:51.173827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 858/1721] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:50 2022] DELETE /identity/v3/projects/2f3bfabfe85447c785cb9cc719688632 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:51.176773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:51.178045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 864/1722] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:51.187352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:51.195796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8bba0-80d1-45d8-b09d-6dfb3e0fcac2 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.196204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8bba0-80d1-45d8-b09d-6dfb3e0fcac2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.200752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8bba0-80d1-45d8-b09d-6dfb3e0fcac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.282788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-22a8bba0-80d1-45d8-b09d-6dfb3e0fcac2 None None] MFA Rules not processed for user `792808dcbf894aa98c671cf43ac21b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:51.291115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-22a8bba0-80d1-45d8-b09d-6dfb3e0fcac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:51.410250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790570b-84a0-466f-9ecd-48e7fa22d0fa admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:51.422252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fba809-7c5d-4767-b167-5560af496cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:51.430184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fba809-7c5d-4767-b167-5560af496cab admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.430184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fba809-7c5d-4767-b167-5560af496cab admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.431975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fba809-7c5d-4767-b167-5560af496cab admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.440175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fba809-7c5d-4767-b167-5560af496cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:51.455274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fba809-7c5d-4767-b167-5560af496cab admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:51.476265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 859/1723] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:51.484420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286cd74-53b1-4610-b6df-f272b0cc3555 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.484705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286cd74-53b1-4610-b6df-f272b0cc3555 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.484977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286cd74-53b1-4610-b6df-f272b0cc3555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.536767 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 865/1724] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:51.539964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f9c4-d625-4ccd-8339-27662df4c08b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.540513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f9c4-d625-4ccd-8339-27662df4c08b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.546183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f9c4-d625-4ccd-8339-27662df4c08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.601593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b286cd74-53b1-4610-b6df-f272b0cc3555 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:51.603279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b286cd74-53b1-4610-b6df-f272b0cc3555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:51.628285 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0484f9c4-d625-4ccd-8339-27662df4c08b None None] MFA Rules not processed for user `eca89f435299415c8448931e73f5b0fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:51.637910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0484f9c4-d625-4ccd-8339-27662df4c08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:51.665150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 860/1725] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:51.687455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286cd74-53b1-4610-b6df-f272b0cc3555 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:51.739035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286cd74-53b1-4610-b6df-f272b0cc3555 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:51.759985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b1093-1eb0-4629-a8c8-55ae36b9e258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:51.760919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b1093-1eb0-4629-a8c8-55ae36b9e258 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.765154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b1093-1eb0-4629-a8c8-55ae36b9e258 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.765595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b1093-1eb0-4629-a8c8-55ae36b9e258 admin admin] PATH_INFO: `/v3/projects/4086e880f09a4f71a3a44efa4a5611bb/users/259683c782a1439ab402bdc3bf7caed8/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.782765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b1093-1eb0-4629-a8c8-55ae36b9e258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4086e880f09a4f71a3a44efa4a5611bb, user_id=259683c782a1439ab402bdc3bf7caed8, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:51.798047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b1093-1eb0-4629-a8c8-55ae36b9e258 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:51.847809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 861/1726] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:51 2022] PUT /identity/v3/projects/4086e880f09a4f71a3a44efa4a5611bb/users/259683c782a1439ab402bdc3bf7caed8/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:51.851348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d33fc-2a18-47b1-9f1c-92f1d405f70e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.851683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d33fc-2a18-47b1-9f1c-92f1d405f70e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.851962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d33fc-2a18-47b1-9f1c-92f1d405f70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.876437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 866/1727] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:51.876437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80f37a-c854-41b7-a1d7-c070346ad956 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.876437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80f37a-c854-41b7-a1d7-c070346ad956 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.876437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80f37a-c854-41b7-a1d7-c070346ad956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.915046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-be2d33fc-2a18-47b1-9f1c-92f1d405f70e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:51.916497 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-be2d33fc-2a18-47b1-9f1c-92f1d405f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:51.975747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:51.994734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-7c80f37a-c854-41b7-a1d7-c070346ad956 None None] MFA Rules not processed for user `259683c782a1439ab402bdc3bf7caed8`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:51.994734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7c80f37a-c854-41b7-a1d7-c070346ad956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:52.075614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 862/1728] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:52.077548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d33fc-2a18-47b1-9f1c-92f1d405f70e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.088015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d33fc-2a18-47b1-9f1c-92f1d405f70e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.093625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.094418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.094954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.095297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.096882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:52.097291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:52.099109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:52.128667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 863/1729] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:52.131934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.135657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 867/1730] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:52.150905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80f37a-c854-41b7-a1d7-c070346ad956 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.152922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7c80f37a-c854-41b7-a1d7-c070346ad956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:52.158887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac35c1d-ece8-4ca1-8c29-7f929ab0c4b9 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.171953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5779c-80ea-4be3-90a5-5c7fdccb6b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.172955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5779c-80ea-4be3-90a5-5c7fdccb6b45 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.174369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5779c-80ea-4be3-90a5-5c7fdccb6b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.174802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5779c-80ea-4be3-90a5-5c7fdccb6b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.175488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5779c-80ea-4be3-90a5-5c7fdccb6b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:52.179269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5779c-80ea-4be3-90a5-5c7fdccb6b45 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:52.301086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80f37a-c854-41b7-a1d7-c070346ad956 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.305706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f5c2b-e683-4262-b489-283d609da716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.306515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f5c2b-e683-4262-b489-283d609da716 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.307204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f5c2b-e683-4262-b489-283d609da716 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.307204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f5c2b-e683-4262-b489-283d609da716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.322168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f5c2b-e683-4262-b489-283d609da716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:52.322168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f5c2b-e683-4262-b489-283d609da716 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:52.349046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 868/1731] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:52.391147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 864/1732] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:52 2022] POST /identity/v3/projects => generated 405 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:52.392475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f5c2b-e683-4262-b489-283d609da716 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.404561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:52.424145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f5c2b-e683-4262-b489-283d609da716 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.585015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef42f8-047c-4137-8a85-6afa7e2665b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.585812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef42f8-047c-4137-8a85-6afa7e2665b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.586043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef42f8-047c-4137-8a85-6afa7e2665b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.586308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef42f8-047c-4137-8a85-6afa7e2665b3 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.587107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef42f8-047c-4137-8a85-6afa7e2665b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:52.589893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef42f8-047c-4137-8a85-6afa7e2665b3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:52.628648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5779c-80ea-4be3-90a5-5c7fdccb6b45 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.705462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5779c-80ea-4be3-90a5-5c7fdccb6b45 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.726838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.727477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.727763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.728043 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.735250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:52.768214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 869/1733] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:52 2022] POST /identity/v3/users => generated 303 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:52.768214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef42f8-047c-4137-8a85-6afa7e2665b3 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.789833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef42f8-047c-4137-8a85-6afa7e2665b3 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.805402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30f14f-05b4-4b68-b034-794225b27562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.806321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30f14f-05b4-4b68-b034-794225b27562 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.806699 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30f14f-05b4-4b68-b034-794225b27562 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.806978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30f14f-05b4-4b68-b034-794225b27562 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.807758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30f14f-05b4-4b68-b034-794225b27562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:52.810120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30f14f-05b4-4b68-b034-794225b27562 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:52.866528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 870/1734] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:52.868153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30f14f-05b4-4b68-b034-794225b27562 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.917055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:52.917355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30f14f-05b4-4b68-b034-794225b27562 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.924488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:52.929675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.929675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.929675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.929675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.931663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:52.939995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:52.958669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 865/1735] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:52.959672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 871/1736] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:52.960337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.979790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31658ce6-f5ac-4df7-8886-2260b7af6687 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.986697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51b70e-4ba2-417b-9503-f78aa10e26f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.987786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51b70e-4ba2-417b-9503-f78aa10e26f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.988088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51b70e-4ba2-417b-9503-f78aa10e26f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.989127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51b70e-4ba2-417b-9503-f78aa10e26f5 admin admin] PATH_INFO: `/v3/projects/603446cfc1f241b1a4eda60634facbfc/users/d2fd3d46f66a417f9bedae5f29261ff6/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.015137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51b70e-4ba2-417b-9503-f78aa10e26f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=603446cfc1f241b1a4eda60634facbfc, user_id=d2fd3d46f66a417f9bedae5f29261ff6, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:53.018510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51b70e-4ba2-417b-9503-f78aa10e26f5 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:53.120566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:53.123460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:53.149524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badbca7-bf68-455c-8a74-7103d3d614ce admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:53.154620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:53.155376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.155736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.155985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] PATH_INFO: `/v3/projects/f15dfb8f979045519dd27a1e4254b387` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.157174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15dfb8f979045519dd27a1e4254b387)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:53.166906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:53.180889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 866/1737] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:52 2022] PUT /identity/v3/projects/603446cfc1f241b1a4eda60634facbfc/users/d2fd3d46f66a417f9bedae5f29261ff6/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:53.196037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aff32-c390-4cd7-9f80-073c3b80387c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.196330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aff32-c390-4cd7-9f80-073c3b80387c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.196740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aff32-c390-4cd7-9f80-073c3b80387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.308800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-713aff32-c390-4cd7-9f80-073c3b80387c None None] MFA Rules not processed for user `d2fd3d46f66a417f9bedae5f29261ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:53.311399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-713aff32-c390-4cd7-9f80-073c3b80387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:53.371733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15dfb8f979045519dd27a1e4254b387', 'request_id': 'req-9ca8d53d-d20a-4696-8e88-5492f22c6a31'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.394895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15dfb8f979045519dd27a1e4254b387', 'request_id': 'req-9ca8d53d-d20a-4696-8e88-5492f22c6a31'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.572179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15dfb8f979045519dd27a1e4254b387', 'request_id': 'req-9ca8d53d-d20a-4696-8e88-5492f22c6a31'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.634438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] The token cache is being invalidate because project f15dfb8f979045519dd27a1e4254b387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:53.635805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.639102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9ca8d53d-d20a-4696-8e88-5492f22c6a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.658697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 872/1738] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:53 2022] DELETE /identity/v3/projects/f15dfb8f979045519dd27a1e4254b387 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:53.665523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4172-90e6-422f-b70f-45c250846b7a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.667032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4172-90e6-422f-b70f-45c250846b7a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.668648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4172-90e6-422f-b70f-45c250846b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.765376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 867/1739] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:53 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 570 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:53.774927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713aff32-c390-4cd7-9f80-073c3b80387c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:53.781844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-713aff32-c390-4cd7-9f80-073c3b80387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:53.823268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-821f4172-90e6-422f-b70f-45c250846b7a None None] MFA Rules not processed for user `e9925313d1d446fa81a00cb55348dff7`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:53.827799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-821f4172-90e6-422f-b70f-45c250846b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:53.958656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 873/1740] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:53.960369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713aff32-c390-4cd7-9f80-073c3b80387c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:53.960994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f4172-90e6-422f-b70f-45c250846b7a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:53.969428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-821f4172-90e6-422f-b70f-45c250846b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:54.114684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:54.114684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:54.114684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:54.114684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] PATH_INFO: `/v3/users/8a61606743f14cf0bc01a0b7a3ee6033` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:54.115645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a61606743f14cf0bc01a0b7a3ee6033)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:54.118333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:54.156444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f4172-90e6-422f-b70f-45c250846b7a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:54.161887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:54.171314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73bbd8-6543-44b1-809f-4ccbd0cd14a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:54.171314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73bbd8-6543-44b1-809f-4ccbd0cd14a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:54.171314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73bbd8-6543-44b1-809f-4ccbd0cd14a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:54.171314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73bbd8-6543-44b1-809f-4ccbd0cd14a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:54.171723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73bbd8-6543-44b1-809f-4ccbd0cd14a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:54.172238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73bbd8-6543-44b1-809f-4ccbd0cd14a9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:54.306079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 874/1741] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:53 2022] POST /identity/v3/projects => generated 405 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:54.310467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73bbd8-6543-44b1-809f-4ccbd0cd14a9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:54.382203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a61606743f14cf0bc01a0b7a3ee6033'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.411799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73bbd8-6543-44b1-809f-4ccbd0cd14a9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:54.433975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4575-27e9-4375-89b5-da18aa1601b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:54.435448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4575-27e9-4375-89b5-da18aa1601b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:54.435732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4575-27e9-4375-89b5-da18aa1601b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:54.436010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4575-27e9-4375-89b5-da18aa1601b4 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:54.436658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d4575-27e9-4375-89b5-da18aa1601b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:54.438661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d4575-27e9-4375-89b5-da18aa1601b4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:54.469450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a61606743f14cf0bc01a0b7a3ee6033'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.470265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a61606743f14cf0bc01a0b7a3ee6033'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.471348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a61606743f14cf0bc01a0b7a3ee6033'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.487021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a61606743f14cf0bc01a0b7a3ee6033'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.527523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 868/1742] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:53 2022] DELETE /identity/v3/users/8a61606743f14cf0bc01a0b7a3ee6033 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:54.548093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 875/1743] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:54 2022] POST /identity/v3/users => generated 309 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:54.549436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:54.550915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:54.690489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc226b-9fbf-4cfb-93ac-5ee0d57460cd admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:54.698859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e66e5-7e73-41a6-9629-25f56254a431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:54.700246 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e66e5-7e73-41a6-9629-25f56254a431 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:54.700666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e66e5-7e73-41a6-9629-25f56254a431 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:54.706688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e66e5-7e73-41a6-9629-25f56254a431 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:54.707391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e66e5-7e73-41a6-9629-25f56254a431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:54.709420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e66e5-7e73-41a6-9629-25f56254a431 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:54.728922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 869/1744] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:54.733075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4575-27e9-4375-89b5-da18aa1601b4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:54.749169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4575-27e9-4375-89b5-da18aa1601b4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:54.758028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0e820-017b-4c35-89d7-16e02cd6afdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:54.778189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e820-017b-4c35-89d7-16e02cd6afdc admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:54.779249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e820-017b-4c35-89d7-16e02cd6afdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:54.779684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e820-017b-4c35-89d7-16e02cd6afdc admin admin] PATH_INFO: `/v3/projects/fcad8e461a4d4c4e87c4fde3c2856244/users/1bfb66fbca64429e901eea625ece1910/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:54.818973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0e820-017b-4c35-89d7-16e02cd6afdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcad8e461a4d4c4e87c4fde3c2856244, user_id=1bfb66fbca64429e901eea625ece1910, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:54.823422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0e820-017b-4c35-89d7-16e02cd6afdc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:54.946826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 876/1745] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:54 2022] PUT /identity/v3/projects/fcad8e461a4d4c4e87c4fde3c2856244/users/1bfb66fbca64429e901eea625ece1910/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:54.953155 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e66e5-7e73-41a6-9629-25f56254a431 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:54.984980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e66e5-7e73-41a6-9629-25f56254a431 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:55.155611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a6ad-b4f5-4090-a815-9291681e0ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:55.156256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a6ad-b4f5-4090-a815-9291681e0ceb admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:55.156539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a6ad-b4f5-4090-a815-9291681e0ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:55.156809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a6ad-b4f5-4090-a815-9291681e0ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:55.157999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a6ad-b4f5-4090-a815-9291681e0ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:55.160154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a6ad-b4f5-4090-a815-9291681e0ceb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:55.164188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:55.181690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 870/1746] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:55.186899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0e820-017b-4c35-89d7-16e02cd6afdc admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:55.202320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0e820-017b-4c35-89d7-16e02cd6afdc admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:55.215784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:55.215784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:55.215784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:55.215784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] PATH_INFO: `/v3/projects/fcad8e461a4d4c4e87c4fde3c2856244/users/1bfb66fbca64429e901eea625ece1910/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:55.227546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcad8e461a4d4c4e87c4fde3c2856244, user_id=1bfb66fbca64429e901eea625ece1910, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:55.230736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:55.278714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 877/1747] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:55 2022] PUT /identity/v3/projects/fcad8e461a4d4c4e87c4fde3c2856244/users/1bfb66fbca64429e901eea625ece1910/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:55.292581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bff4d-182f-4ade-8f6e-f7a97ad69957 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:55.292944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bff4d-182f-4ade-8f6e-f7a97ad69957 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:55.293284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bff4d-182f-4ade-8f6e-f7a97ad69957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:55.388657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-a49bff4d-182f-4ade-8f6e-f7a97ad69957 None None] MFA Rules not processed for user `1bfb66fbca64429e901eea625ece1910`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:55.389985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a49bff4d-182f-4ade-8f6e-f7a97ad69957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:55.517960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 871/1748] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:55 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:55.719865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:55.727827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:55.831624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d00276-990f-4896-9df6-ca75bdddad40 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:55.844116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:55.844774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:55.847231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:55.847538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] PATH_INFO: `/v3/projects/3d4fd87e7e0543eb877d77168438a74d` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:55.851164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4fd87e7e0543eb877d77168438a74d)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:55.857521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:55.981872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bff4d-182f-4ade-8f6e-f7a97ad69957 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:55.983320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a49bff4d-182f-4ade-8f6e-f7a97ad69957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.055998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:56.062919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4fd87e7e0543eb877d77168438a74d', 'request_id': 'req-95d6d14c-d415-4776-9c79-7942ae828ac4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:56.085778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4fd87e7e0543eb877d77168438a74d', 'request_id': 'req-95d6d14c-d415-4776-9c79-7942ae828ac4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:56.103726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bff4d-182f-4ade-8f6e-f7a97ad69957 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.120110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:56.184738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.185366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.185717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.186061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.191970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4fd87e7e0543eb877d77168438a74d', 'request_id': 'req-95d6d14c-d415-4776-9c79-7942ae828ac4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:56.206990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] The token cache is being invalidate because project 3d4fd87e7e0543eb877d77168438a74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:56.207648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:56.209503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95d6d14c-d415-4776-9c79-7942ae828ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:56.216744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 878/1749] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:55 2022] DELETE /identity/v3/projects/3d4fd87e7e0543eb877d77168438a74d => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:56.218910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad1182-6f97-44ce-af2b-791ca0bf6659 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.219410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad1182-6f97-44ce-af2b-791ca0bf6659 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.220844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad1182-6f97-44ce-af2b-791ca0bf6659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.247242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-b8ad1182-6f97-44ce-af2b-791ca0bf6659 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:56.248855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b8ad1182-6f97-44ce-af2b-791ca0bf6659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.369173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 879/1750] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:56.370976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c87080-e68c-4b34-97e5-117ec6c685f0 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.371431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c87080-e68c-4b34-97e5-117ec6c685f0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.371927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c87080-e68c-4b34-97e5-117ec6c685f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.389364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.390878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.429930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 872/1751] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:55 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:56.430568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-28c87080-e68c-4b34-97e5-117ec6c685f0 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:56.432945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-28c87080-e68c-4b34-97e5-117ec6c685f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.452166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 880/1752] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:56.452287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.453704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.546118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292028a-b220-4f8c-ab37-214a4d04e41b service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.567376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a12d24-b232-466c-8b55-49f31a57514b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.568287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a12d24-b232-466c-8b55-49f31a57514b admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.568738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a12d24-b232-466c-8b55-49f31a57514b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.574172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a12d24-b232-466c-8b55-49f31a57514b admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.576846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a12d24-b232-466c-8b55-49f31a57514b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:56.576846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a12d24-b232-466c-8b55-49f31a57514b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.577631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a12d24-b232-466c-8b55-49f31a57514b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.615449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c87080-e68c-4b34-97e5-117ec6c685f0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.616016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 873/1753] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:56.661071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c87080-e68c-4b34-97e5-117ec6c685f0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.707752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5637d0-79b2-4dc2-8930-bf10335df99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.707752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5637d0-79b2-4dc2-8930-bf10335df99d admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.707752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5637d0-79b2-4dc2-8930-bf10335df99d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.707752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5637d0-79b2-4dc2-8930-bf10335df99d admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.708230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5637d0-79b2-4dc2-8930-bf10335df99d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.710607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5637d0-79b2-4dc2-8930-bf10335df99d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.713677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c3be9-fa5a-424b-b3fe-1ffedb11ad6e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.713940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c3be9-fa5a-424b-b3fe-1ffedb11ad6e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.714293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c3be9-fa5a-424b-b3fe-1ffedb11ad6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.926255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-7f4c3be9-fa5a-424b-b3fe-1ffedb11ad6e None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:56.927754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7f4c3be9-fa5a-424b-b3fe-1ffedb11ad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.943114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 881/1754] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/projects => generated 399 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:56.948253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5637d0-79b2-4dc2-8930-bf10335df99d admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.041163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 874/1755] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:57.042568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c3be9-fa5a-424b-b3fe-1ffedb11ad6e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.043967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7f4c3be9-fa5a-424b-b3fe-1ffedb11ad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:57.079355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c3be9-fa5a-424b-b3fe-1ffedb11ad6e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.158427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5637d0-79b2-4dc2-8930-bf10335df99d admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.171634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b192ec8-9705-47c1-8f84-fe5cc32ea456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.172404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b192ec8-9705-47c1-8f84-fe5cc32ea456 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.172688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b192ec8-9705-47c1-8f84-fe5cc32ea456 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.173139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b192ec8-9705-47c1-8f84-fe5cc32ea456 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.173783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b192ec8-9705-47c1-8f84-fe5cc32ea456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.176025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b192ec8-9705-47c1-8f84-fe5cc32ea456 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.198079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.198821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.199097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.199353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] PATH_INFO: `/v3/users/d2fd3d46f66a417f9bedae5f29261ff6` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.203234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2fd3d46f66a417f9bedae5f29261ff6)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.206053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.330406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 882/1756] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/users => generated 300 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:57.336165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b192ec8-9705-47c1-8f84-fe5cc32ea456 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.358504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b192ec8-9705-47c1-8f84-fe5cc32ea456 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.369515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc8f17-8905-4dab-8cdd-d789c5534c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.370937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8f17-8905-4dab-8cdd-d789c5534c8a admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.371280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8f17-8905-4dab-8cdd-d789c5534c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.372147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8f17-8905-4dab-8cdd-d789c5534c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.372947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc8f17-8905-4dab-8cdd-d789c5534c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.374666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc8f17-8905-4dab-8cdd-d789c5534c8a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.385676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 883/1757] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:57.390363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc8f17-8905-4dab-8cdd-d789c5534c8a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.401784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc8f17-8905-4dab-8cdd-d789c5534c8a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.408938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2205156-cffa-4892-9e90-7935768dcda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.409757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2205156-cffa-4892-9e90-7935768dcda7 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.410624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2205156-cffa-4892-9e90-7935768dcda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.410911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2205156-cffa-4892-9e90-7935768dcda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.411576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2205156-cffa-4892-9e90-7935768dcda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.413421 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2205156-cffa-4892-9e90-7935768dcda7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.425438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 884/1758] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:57.430217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2205156-cffa-4892-9e90-7935768dcda7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.442495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2205156-cffa-4892-9e90-7935768dcda7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.464933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.464933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.464933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.464933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] PATH_INFO: `/v3/projects/9425cc16384c4f16a92889f135ad0b39/users/8073986688d44b0aba17723877cde6c0/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.465398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2fd3d46f66a417f9bedae5f29261ff6'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:57.486182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9425cc16384c4f16a92889f135ad0b39, user_id=8073986688d44b0aba17723877cde6c0, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.489271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.569229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2fd3d46f66a417f9bedae5f29261ff6'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:57.570460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2fd3d46f66a417f9bedae5f29261ff6'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:57.571217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2fd3d46f66a417f9bedae5f29261ff6'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:57.588507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b24a4150-b3d0-44a9-8e7a-f2128fad9caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2fd3d46f66a417f9bedae5f29261ff6'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:57.640467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 885/1759] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:57 2022] PUT /identity/v3/projects/9425cc16384c4f16a92889f135ad0b39/users/8073986688d44b0aba17723877cde6c0/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:57.642072 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.645052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:57.674998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 875/1760] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:57 2022] DELETE /identity/v3/users/d2fd3d46f66a417f9bedae5f29261ff6 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:57.681991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4aaf0-997b-4089-b128-6f0a0a7d5c7b None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.682683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4aaf0-997b-4089-b128-6f0a0a7d5c7b None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.685895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4aaf0-997b-4089-b128-6f0a0a7d5c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.781938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-59d4aaf0-997b-4089-b128-6f0a0a7d5c7b None None] MFA Rules not processed for user `8073986688d44b0aba17723877cde6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:57.783667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-59d4aaf0-997b-4089-b128-6f0a0a7d5c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:57.870882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d254b-0460-4c26-a84f-6a70dcedbe72 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:58.037209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 876/1761] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:58.038786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27fe4-0039-4a22-8255-bb3ae5193492 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.039631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27fe4-0039-4a22-8255-bb3ae5193492 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.040095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27fe4-0039-4a22-8255-bb3ae5193492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.071720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-16e27fe4-0039-4a22-8255-bb3ae5193492 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:58.073340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-16e27fe4-0039-4a22-8255-bb3ae5193492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:58.104292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf4df6-5972-4533-b4a3-c2c9f4b6008b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:58.104933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4df6-5972-4533-b4a3-c2c9f4b6008b service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.105387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4df6-5972-4533-b4a3-c2c9f4b6008b service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.108798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4df6-5972-4533-b4a3-c2c9f4b6008b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.120559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-73bf4df6-5972-4533-b4a3-c2c9f4b6008b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:58.281222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4df6-5972-4533-b4a3-c2c9f4b6008b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:58.283000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4df6-5972-4533-b4a3-c2c9f4b6008b service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:58.306091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 886/1762] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 13:53:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:53:58.307049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204a723-97c7-411e-8359-33266a44e134 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.309934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204a723-97c7-411e-8359-33266a44e134 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.311646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204a723-97c7-411e-8359-33266a44e134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.327301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 877/1763] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:58.328198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27fe4-0039-4a22-8255-bb3ae5193492 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:58.339944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-16e27fe4-0039-4a22-8255-bb3ae5193492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:58.364729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0204a723-97c7-411e-8359-33266a44e134 None None] MFA Rules not processed for user `a71d353c1bd849028930261e9306fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:58.371180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0204a723-97c7-411e-8359-33266a44e134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:58.374844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27fe4-0039-4a22-8255-bb3ae5193492 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:58.395060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:58.396446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.396638 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.397151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.397992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:58.399996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:58.468900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:58.689971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 878/1764] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:58 2022] POST /identity/v3/projects => generated 399 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:58.689971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:58.689971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:58.691806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 887/1765] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Sep 8 13:53:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:58.718501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204a723-97c7-411e-8359-33266a44e134 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:58.764218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204a723-97c7-411e-8359-33266a44e134 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:58.914467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cede3c-8366-4d39-b63f-6f323f1581b8 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:58.927057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6aa3a-d8d1-4b63-9d6e-1a931bdeb088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:58.928460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6aa3a-d8d1-4b63-9d6e-1a931bdeb088 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.929669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6aa3a-d8d1-4b63-9d6e-1a931bdeb088 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.930038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6aa3a-d8d1-4b63-9d6e-1a931bdeb088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.977846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7dc45-719d-442c-a875-f16fccd67ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:58.978567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dc45-719d-442c-a875-f16fccd67ccd admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.978926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dc45-719d-442c-a875-f16fccd67ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.979269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dc45-719d-442c-a875-f16fccd67ccd admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.980016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7dc45-719d-442c-a875-f16fccd67ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:58.985865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7dc45-719d-442c-a875-f16fccd67ccd admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.099358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6aa3a-d8d1-4b63-9d6e-1a931bdeb088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.101165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6aa3a-d8d1-4b63-9d6e-1a931bdeb088 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.119763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 879/1766] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:59.120734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6aa3a-d8d1-4b63-9d6e-1a931bdeb088 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:59.153910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 888/1767] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:58 2022] POST /identity/v3/users => generated 306 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:59.160266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6aa3a-d8d1-4b63-9d6e-1a931bdeb088 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:59.161247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7dc45-719d-442c-a875-f16fccd67ccd admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:59.165466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594054b4-4fde-4a25-aafc-8b43e2522aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:59.165701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594054b4-4fde-4a25-aafc-8b43e2522aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.165990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594054b4-4fde-4a25-aafc-8b43e2522aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.166285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594054b4-4fde-4a25-aafc-8b43e2522aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.196865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7dc45-719d-442c-a875-f16fccd67ccd admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:59.207701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31490e-89e2-4574-b103-42358c026ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:59.208197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594054b4-4fde-4a25-aafc-8b43e2522aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.209691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594054b4-4fde-4a25-aafc-8b43e2522aa8 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.211745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31490e-89e2-4574-b103-42358c026ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.212298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31490e-89e2-4574-b103-42358c026ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.212775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31490e-89e2-4574-b103-42358c026ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.213695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31490e-89e2-4574-b103-42358c026ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.217559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31490e-89e2-4574-b103-42358c026ce8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.236222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 880/1768] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:59.242486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 889/1769] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:59.243723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594054b4-4fde-4a25-aafc-8b43e2522aa8 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:59.280858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594054b4-4fde-4a25-aafc-8b43e2522aa8 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:59.325980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bb51d-ca0e-4827-aea1-6b14988ad003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:59.328399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bb51d-ca0e-4827-aea1-6b14988ad003 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.328674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bb51d-ca0e-4827-aea1-6b14988ad003 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.328950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bb51d-ca0e-4827-aea1-6b14988ad003 admin admin] PATH_INFO: `/v3/projects/2da1ee3697554d01b14074cc55ffe49b/users/5a668c884ef94611b148077460bba608/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.401207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bb51d-ca0e-4827-aea1-6b14988ad003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da1ee3697554d01b14074cc55ffe49b, user_id=5a668c884ef94611b148077460bba608, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.409202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bb51d-ca0e-4827-aea1-6b14988ad003 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.524393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 881/1770] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:59 2022] PUT /identity/v3/projects/2da1ee3697554d01b14074cc55ffe49b/users/5a668c884ef94611b148077460bba608/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:59.530484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31490e-89e2-4574-b103-42358c026ce8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:59.547619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31490e-89e2-4574-b103-42358c026ce8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:59.603555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:59.764494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744a32d-921e-4b2d-a6da-8f7bef16ba2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:59.765499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744a32d-921e-4b2d-a6da-8f7bef16ba2e admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.765974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744a32d-921e-4b2d-a6da-8f7bef16ba2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.766473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744a32d-921e-4b2d-a6da-8f7bef16ba2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.767408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744a32d-921e-4b2d-a6da-8f7bef16ba2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.769266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744a32d-921e-4b2d-a6da-8f7bef16ba2e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.810539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 890/1771] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:59.810708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bb51d-ca0e-4827-aea1-6b14988ad003 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:59.826587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bb51d-ca0e-4827-aea1-6b14988ad003 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:59.864810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:59.865738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.866246 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.866685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] PATH_INFO: `/v3/projects/2da1ee3697554d01b14074cc55ffe49b/users/5a668c884ef94611b148077460bba608/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.870765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da1ee3697554d01b14074cc55ffe49b, user_id=5a668c884ef94611b148077460bba608, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.876680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.934413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 882/1772] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:59 2022] PUT /identity/v3/projects/2da1ee3697554d01b14074cc55ffe49b/users/5a668c884ef94611b148077460bba608/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:59.937112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837b8c5-e239-4fb4-853d-0fa631916786 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.937996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837b8c5-e239-4fb4-853d-0fa631916786 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.938581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837b8c5-e239-4fb4-853d-0fa631916786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.008716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a837b8c5-e239-4fb4-853d-0fa631916786 None None] MFA Rules not processed for user `5a668c884ef94611b148077460bba608`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:00.013872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a837b8c5-e239-4fb4-853d-0fa631916786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:00.131276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 891/1773] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:53:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:00.150605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:00.153662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:00.268348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19c056-1986-4031-a565-c8cb5c90286f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:00.278496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:00.280059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:00.280059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:00.280059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.293393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:00.295100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:00.316119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 883/1774] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:00 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:00.629309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837b8c5-e239-4fb4-853d-0fa631916786 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:00.640830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a837b8c5-e239-4fb4-853d-0fa631916786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:00.656437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837b8c5-e239-4fb4-853d-0fa631916786 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:00.660591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce8094-8a5f-43e3-ae51-60d4ff872452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:00.661453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce8094-8a5f-43e3-ae51-60d4ff872452 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:00.661736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce8094-8a5f-43e3-ae51-60d4ff872452 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:00.662010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce8094-8a5f-43e3-ae51-60d4ff872452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.711052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:00.712642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:00.739826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce8094-8a5f-43e3-ae51-60d4ff872452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:00.741669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce8094-8a5f-43e3-ae51-60d4ff872452 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:00.755332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 892/1775] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:00.833303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceadd7-df53-4dff-8a6d-6bb3a792c4f9 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:00.840105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:00.840981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:00.841423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:00.841858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] PATH_INFO: `/v3/projects/603446cfc1f241b1a4eda60634facbfc` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.843308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] RBAC: Authorizing `identity:delete_project(project_id=603446cfc1f241b1a4eda60634facbfc)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:00.845680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:00.985744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603446cfc1f241b1a4eda60634facbfc', 'request_id': 'req-eae037db-a1ea-47cf-b7e5-dc256c57f29b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:00.996666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603446cfc1f241b1a4eda60634facbfc', 'request_id': 'req-eae037db-a1ea-47cf-b7e5-dc256c57f29b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.097981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603446cfc1f241b1a4eda60634facbfc', 'request_id': 'req-eae037db-a1ea-47cf-b7e5-dc256c57f29b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.115119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] The token cache is being invalidate because project 603446cfc1f241b1a4eda60634facbfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:01.115791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.116665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eae037db-a1ea-47cf-b7e5-dc256c57f29b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.118732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 884/1776] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:00 2022] DELETE /identity/v3/projects/603446cfc1f241b1a4eda60634facbfc => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:01.123915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce8094-8a5f-43e3-ae51-60d4ff872452 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:01.136996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce8094-8a5f-43e3-ae51-60d4ff872452 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:01.197745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:01.198523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.198806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.199081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] PATH_INFO: `/v3/users/1bfb66fbca64429e901eea625ece1910` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.200273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bfb66fbca64429e901eea625ece1910)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:01.202227 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:01.428338 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bfb66fbca64429e901eea625ece1910'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.458302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1239b3-e5ac-44ff-b94d-795be25ff980 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.458588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1239b3-e5ac-44ff-b94d-795be25ff980 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.459089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1239b3-e5ac-44ff-b94d-795be25ff980 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.460505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 885/1777] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:01.482853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf4c1-a83b-479d-957a-2c2996932867 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.483539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf4c1-a83b-479d-957a-2c2996932867 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.484030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf4c1-a83b-479d-957a-2c2996932867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.507429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bfb66fbca64429e901eea625ece1910'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.511823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bfb66fbca64429e901eea625ece1910'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.511979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-18ccf4c1-a83b-479d-957a-2c2996932867 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:01.514957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-18ccf4c1-a83b-479d-957a-2c2996932867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:01.595451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bfb66fbca64429e901eea625ece1910'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.609468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bfb66fbca64429e901eea625ece1910'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:01.638486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 886/1778] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:54:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:01.643210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 893/1779] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:01 2022] DELETE /identity/v3/users/1bfb66fbca64429e901eea625ece1910 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:01.697140 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:01.802783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf4c1-a83b-479d-957a-2c2996932867 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:01.804215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-18ccf4c1-a83b-479d-957a-2c2996932867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:01.818594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf4c1-a83b-479d-957a-2c2996932867 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:01.885572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba795923-2730-4341-9618-19b39cd1818d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:01.886232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba795923-2730-4341-9618-19b39cd1818d service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.886520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba795923-2730-4341-9618-19b39cd1818d service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.886801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba795923-2730-4341-9618-19b39cd1818d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.896085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba795923-2730-4341-9618-19b39cd1818d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:01.897715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba795923-2730-4341-9618-19b39cd1818d service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:01.908873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 887/1780] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:02.178771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:02.179180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:02.312138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13aa95-a498-4547-a06f-9cfd4e6ad96a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:02.317545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:02.317665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:02.317915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:02.318440 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] PATH_INFO: `/v3/projects/fcad8e461a4d4c4e87c4fde3c2856244` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:02.319796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcad8e461a4d4c4e87c4fde3c2856244)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:02.321964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:02.455895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcad8e461a4d4c4e87c4fde3c2856244', 'request_id': 'req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:02.463393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcad8e461a4d4c4e87c4fde3c2856244', 'request_id': 'req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:02.467225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:02.588618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcad8e461a4d4c4e87c4fde3c2856244', 'request_id': 'req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:02.605798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] The token cache is being invalidate because project fcad8e461a4d4c4e87c4fde3c2856244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:02.606342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:02.607141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef8e9d59-fb71-42e5-a42a-af2adf8c6c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:02.609283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 894/1781] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:02 2022] DELETE /identity/v3/projects/fcad8e461a4d4c4e87c4fde3c2856244 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:02.614303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade69d2e-9b6d-4b03-83b8-071bde6d4769 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:02.614602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade69d2e-9b6d-4b03-83b8-071bde6d4769 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:02.614886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade69d2e-9b6d-4b03-83b8-071bde6d4769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:02.648585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ade69d2e-9b6d-4b03-83b8-071bde6d4769 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:02.650399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ade69d2e-9b6d-4b03-83b8-071bde6d4769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:02.765969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 888/1782] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:02.774199 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335d1ff-f2d1-4555-9f13-f646f68e57dd None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:02.775258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335d1ff-f2d1-4555-9f13-f646f68e57dd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:02.775752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335d1ff-f2d1-4555-9f13-f646f68e57dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:02.828173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-2335d1ff-f2d1-4555-9f13-f646f68e57dd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:02.831469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2335d1ff-f2d1-4555-9f13-f646f68e57dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:02.853784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 895/1783] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:02.858881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade69d2e-9b6d-4b03-83b8-071bde6d4769 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:02.874922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade69d2e-9b6d-4b03-83b8-071bde6d4769 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:02.881925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:02.883546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:02.883546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:02.883546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:02.884696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:02.885171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:02.887130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:02.914551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 889/1784] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:02.917262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335d1ff-f2d1-4555-9f13-f646f68e57dd None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:02.928838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335d1ff-f2d1-4555-9f13-f646f68e57dd None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:02.943375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee1aee-09ed-4c1c-b024-cb2b2b48b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:02.944224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee1aee-09ed-4c1c-b024-cb2b2b48b7fc admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:02.944686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee1aee-09ed-4c1c-b024-cb2b2b48b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:02.945110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee1aee-09ed-4c1c-b024-cb2b2b48b7fc admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:02.946011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee1aee-09ed-4c1c-b024-cb2b2b48b7fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:02.948115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee1aee-09ed-4c1c-b024-cb2b2b48b7fc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.069441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 896/1785] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:02 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:03.072762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.093490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960a086-1a93-47dd-8c2c-62230299903b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:03.194692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33857f9f-1f6f-44bd-8131-1ccf9b6ba4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:03.195341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33857f9f-1f6f-44bd-8131-1ccf9b6ba4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.195677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33857f9f-1f6f-44bd-8131-1ccf9b6ba4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.195931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33857f9f-1f6f-44bd-8131-1ccf9b6ba4a0 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.196593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33857f9f-1f6f-44bd-8131-1ccf9b6ba4a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:03.198570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33857f9f-1f6f-44bd-8131-1ccf9b6ba4a0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.502236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:03.512853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 890/1786] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:03 2022] POST /identity/v3/users => generated 291 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:03.514545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee1aee-09ed-4c1c-b024-cb2b2b48b7fc admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.529928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee1aee-09ed-4c1c-b024-cb2b2b48b7fc admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:03.541794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314666a-c0b2-4dfe-9e21-c2ff631f5fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:03.542872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314666a-c0b2-4dfe-9e21-c2ff631f5fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.543421 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314666a-c0b2-4dfe-9e21-c2ff631f5fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.544637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314666a-c0b2-4dfe-9e21-c2ff631f5fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.545453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314666a-c0b2-4dfe-9e21-c2ff631f5fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:03.547335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314666a-c0b2-4dfe-9e21-c2ff631f5fa2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.579552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 897/1787] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:03.579981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33857f9f-1f6f-44bd-8131-1ccf9b6ba4a0 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.580090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33857f9f-1f6f-44bd-8131-1ccf9b6ba4a0 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:03.585915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af8229-25b1-4928-9b7e-0a525d88808a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:03.586707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af8229-25b1-4928-9b7e-0a525d88808a admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.587099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af8229-25b1-4928-9b7e-0a525d88808a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.587498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af8229-25b1-4928-9b7e-0a525d88808a admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.592438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af8229-25b1-4928-9b7e-0a525d88808a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:03.594569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af8229-25b1-4928-9b7e-0a525d88808a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.609940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 891/1788] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:03.615858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314666a-c0b2-4dfe-9e21-c2ff631f5fa2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.643628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314666a-c0b2-4dfe-9e21-c2ff631f5fa2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:03.653753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bd586-554e-4075-8d4d-579522d1f269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:03.654802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bd586-554e-4075-8d4d-579522d1f269 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.655137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bd586-554e-4075-8d4d-579522d1f269 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.655378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bd586-554e-4075-8d4d-579522d1f269 admin admin] PATH_INFO: `/v3/projects/fb5fb98e61d54e9fbded87e523e1c46e/users/a714cc954cf541988bf11f3fe91e6e24/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.688500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bd586-554e-4075-8d4d-579522d1f269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5fb98e61d54e9fbded87e523e1c46e, user_id=a714cc954cf541988bf11f3fe91e6e24, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:03.691759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bd586-554e-4075-8d4d-579522d1f269 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.770879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:03.819374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 898/1789] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:03 2022] PUT /identity/v3/projects/fb5fb98e61d54e9fbded87e523e1c46e/users/a714cc954cf541988bf11f3fe91e6e24/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:03.958542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76fc0-962b-4731-a431-13c4f7ccdff6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.958542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76fc0-962b-4731-a431-13c4f7ccdff6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.958542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb76fc0-962b-4731-a431-13c4f7ccdff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.958542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-deb76fc0-962b-4731-a431-13c4f7ccdff6 None None] MFA Rules not processed for user `a714cc954cf541988bf11f3fe91e6e24`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:03.958542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-deb76fc0-962b-4731-a431-13c4f7ccdff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.968007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 892/1790] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:03.974306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bd586-554e-4075-8d4d-579522d1f269 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.080867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bd586-554e-4075-8d4d-579522d1f269 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.085997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328ee45-82ea-4a65-9163-a4bf0fbe7ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.086493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328ee45-82ea-4a65-9163-a4bf0fbe7ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.086781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328ee45-82ea-4a65-9163-a4bf0fbe7ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.087057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328ee45-82ea-4a65-9163-a4bf0fbe7ea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.087690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328ee45-82ea-4a65-9163-a4bf0fbe7ea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.089793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328ee45-82ea-4a65-9163-a4bf0fbe7ea3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.250211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 899/1791] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:03 2022] POST /identity/v3/projects => generated 381 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:04.255112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb76fc0-962b-4731-a431-13c4f7ccdff6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.277378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb76fc0-962b-4731-a431-13c4f7ccdff6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.427432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12b47e-797e-4f49-adf1-7dc3bbdefdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.427432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12b47e-797e-4f49-adf1-7dc3bbdefdaa admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.427432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12b47e-797e-4f49-adf1-7dc3bbdefdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.427432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12b47e-797e-4f49-adf1-7dc3bbdefdaa admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.427787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12b47e-797e-4f49-adf1-7dc3bbdefdaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.427787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12b47e-797e-4f49-adf1-7dc3bbdefdaa admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.683754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 893/1792] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:04 2022] POST /identity/v3/users => generated 297 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:04.688110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328ee45-82ea-4a65-9163-a4bf0fbe7ea3 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.702458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328ee45-82ea-4a65-9163-a4bf0fbe7ea3 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.719832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890366f-2a00-493e-9b0e-178b6a2a01e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.719832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890366f-2a00-493e-9b0e-178b6a2a01e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.720226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890366f-2a00-493e-9b0e-178b6a2a01e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.720781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890366f-2a00-493e-9b0e-178b6a2a01e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.721728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890366f-2a00-493e-9b0e-178b6a2a01e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.725238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890366f-2a00-493e-9b0e-178b6a2a01e8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.740869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 900/1793] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:04.746046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12b47e-797e-4f49-adf1-7dc3bbdefdaa admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.759221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12b47e-797e-4f49-adf1-7dc3bbdefdaa admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.764845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a680ff-ee03-421f-b90e-d77e292f420c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.765929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a680ff-ee03-421f-b90e-d77e292f420c admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.766444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a680ff-ee03-421f-b90e-d77e292f420c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.766798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a680ff-ee03-421f-b90e-d77e292f420c admin admin] PATH_INFO: `/v3/projects/69ffb76b6f574f1bab8bc5cbd7ac9d95/users/caae89653f244fe7aaad4a8fa0150721/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.807069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a680ff-ee03-421f-b90e-d77e292f420c admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ffb76b6f574f1bab8bc5cbd7ac9d95, user_id=caae89653f244fe7aaad4a8fa0150721, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.814311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a680ff-ee03-421f-b90e-d77e292f420c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:05.147038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 894/1794] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:04 2022] PUT /identity/v3/projects/69ffb76b6f574f1bab8bc5cbd7ac9d95/users/caae89653f244fe7aaad4a8fa0150721/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:05.150676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890366f-2a00-493e-9b0e-178b6a2a01e8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:05.163347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890366f-2a00-493e-9b0e-178b6a2a01e8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:05.254200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e122710-347a-4c9c-aedf-09af2b5452d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:05.254200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e122710-347a-4c9c-aedf-09af2b5452d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.254200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e122710-347a-4c9c-aedf-09af2b5452d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.254200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e122710-347a-4c9c-aedf-09af2b5452d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.254533 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e122710-347a-4c9c-aedf-09af2b5452d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:05.256042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e122710-347a-4c9c-aedf-09af2b5452d4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:05.269385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 901/1795] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:05.277717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a680ff-ee03-421f-b90e-d77e292f420c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:05.300486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a680ff-ee03-421f-b90e-d77e292f420c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:05.306416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:05.307004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.307293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.307570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] PATH_INFO: `/v3/projects/69ffb76b6f574f1bab8bc5cbd7ac9d95/users/caae89653f244fe7aaad4a8fa0150721/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.310435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ffb76b6f574f1bab8bc5cbd7ac9d95, user_id=caae89653f244fe7aaad4a8fa0150721, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:05.313471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:05.327239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 895/1796] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:05 2022] PUT /identity/v3/projects/69ffb76b6f574f1bab8bc5cbd7ac9d95/users/caae89653f244fe7aaad4a8fa0150721/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:05.338549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec20443-da08-484e-b43b-cb5d07d0b2fb None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.338894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec20443-da08-484e-b43b-cb5d07d0b2fb None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.339109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec20443-da08-484e-b43b-cb5d07d0b2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.380805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-cec20443-da08-484e-b43b-cb5d07d0b2fb None None] MFA Rules not processed for user `caae89653f244fe7aaad4a8fa0150721`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:05.381153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cec20443-da08-484e-b43b-cb5d07d0b2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:05.497705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 902/1797] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:05.935854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:05.938740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:06.034015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adae89-158c-45ba-885c-673415394e6f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:06.045012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2014f73-95b6-4bbd-8107-f7eaf3dec5f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:06.045653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2014f73-95b6-4bbd-8107-f7eaf3dec5f7 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:06.046116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2014f73-95b6-4bbd-8107-f7eaf3dec5f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:06.046424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2014f73-95b6-4bbd-8107-f7eaf3dec5f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:06.079666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2014f73-95b6-4bbd-8107-f7eaf3dec5f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:06.081606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2014f73-95b6-4bbd-8107-f7eaf3dec5f7 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:06.100180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 896/1798] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:09.550648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec20443-da08-484e-b43b-cb5d07d0b2fb None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.560972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec20443-da08-484e-b43b-cb5d07d0b2fb None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.566295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caae89653f244fe7aaad4a8fa0150721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69ffb76b6f574f1bab8bc5cbd7ac9d95', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.567041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.567415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.567799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.568560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:09.568988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.570884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.583849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 903/1799] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:09.588294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2014f73-95b6-4bbd-8107-f7eaf3dec5f7 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.599065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2014f73-95b6-4bbd-8107-f7eaf3dec5f7 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.604616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183cc69-92fd-4b76-9c3e-d77fc8787569 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caae89653f244fe7aaad4a8fa0150721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69ffb76b6f574f1bab8bc5cbd7ac9d95', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.605287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183cc69-92fd-4b76-9c3e-d77fc8787569 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.605803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183cc69-92fd-4b76-9c3e-d77fc8787569 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.613439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183cc69-92fd-4b76-9c3e-d77fc8787569 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.613439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183cc69-92fd-4b76-9c3e-d77fc8787569 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.613439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183cc69-92fd-4b76-9c3e-d77fc8787569 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.745752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 897/1800] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:09 2022] POST /identity/v3/projects => generated 353 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:09.749812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.760173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1783567-48ca-4583-a4f7-f059fe651846 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.858214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caae89653f244fe7aaad4a8fa0150721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69ffb76b6f574f1bab8bc5cbd7ac9d95', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.859006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.860951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.861404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.862074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:09.864578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.864578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.877937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 904/1801] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:10.032601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183cc69-92fd-4b76-9c3e-d77fc8787569 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:10.046702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183cc69-92fd-4b76-9c3e-d77fc8787569 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:10.053064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caae89653f244fe7aaad4a8fa0150721', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69ffb76b6f574f1bab8bc5cbd7ac9d95', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:10.053926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:10.054255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:10.055143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] PATH_INFO: `/v3/projects/2d147adbe99241f3bd5bd0bb8fecba70` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:10.056493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2d147adbe99241f3bd5bd0bb8fecba70)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:10.059952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:10.177526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:10.180158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:10.211335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d147adbe99241f3bd5bd0bb8fecba70', 'request_id': 'req-51595256-d7da-4229-845d-032f2e5728ab'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.238896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d147adbe99241f3bd5bd0bb8fecba70', 'request_id': 'req-51595256-d7da-4229-845d-032f2e5728ab'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.322624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85602e09-38f1-4bbf-8261-8816c72ab0f5 tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:10.380931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d147adbe99241f3bd5bd0bb8fecba70', 'request_id': 'req-51595256-d7da-4229-845d-032f2e5728ab'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.410682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] The token cache is being invalidate because project 2d147adbe99241f3bd5bd0bb8fecba70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:10.415946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.419423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-51595256-d7da-4229-845d-032f2e5728ab tempest-VolumeQuotaClassesTest-1276012679 tempest-VolumeQuotaClassesTest-1276012679-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.422439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 898/1802] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:10 2022] DELETE /identity/v3/projects/2d147adbe99241f3bd5bd0bb8fecba70 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:10.451292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:10.452319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:10.452601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:10.452879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] PATH_INFO: `/v3/users/8073986688d44b0aba17723877cde6c0` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:10.454330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8073986688d44b0aba17723877cde6c0)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:10.456314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:10.745372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8073986688d44b0aba17723877cde6c0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.839302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8073986688d44b0aba17723877cde6c0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.840068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8073986688d44b0aba17723877cde6c0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.858183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8073986688d44b0aba17723877cde6c0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.866634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8073986688d44b0aba17723877cde6c0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:10.876219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 905/1803] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:10 2022] DELETE /identity/v3/users/8073986688d44b0aba17723877cde6c0 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:10.881729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286c549-4525-4b03-b539-5ab141c661d1 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:10.886823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286c549-4525-4b03-b539-5ab141c661d1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:10.886823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286c549-4525-4b03-b539-5ab141c661d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:10.914423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-7286c549-4525-4b03-b539-5ab141c661d1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:10.916117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7286c549-4525-4b03-b539-5ab141c661d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:11.057105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 899/1804] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:11.069919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:11.071381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:11.218816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0feaee-c955-4bfc-8761-7be63de33bd5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:11.222679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec448c-0ecf-444e-b297-bdf587e37f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:11.223539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec448c-0ecf-444e-b297-bdf587e37f38 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:11.227404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec448c-0ecf-444e-b297-bdf587e37f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:11.227404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec448c-0ecf-444e-b297-bdf587e37f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:11.235490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec448c-0ecf-444e-b297-bdf587e37f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:11.237244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec448c-0ecf-444e-b297-bdf587e37f38 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:11.248911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 906/1805] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:11.362068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286c549-4525-4b03-b539-5ab141c661d1 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:11.365603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7286c549-4525-4b03-b539-5ab141c661d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:11.406934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286c549-4525-4b03-b539-5ab141c661d1 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:11.437523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:11.438658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:11.442590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:11.447248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] PATH_INFO: `/v3/users/a714cc954cf541988bf11f3fe91e6e24` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:11.449193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a714cc954cf541988bf11f3fe91e6e24)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:11.458797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:11.539153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:11.657663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:11.911958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a714cc954cf541988bf11f3fe91e6e24'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:12.063386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a714cc954cf541988bf11f3fe91e6e24'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:12.063386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a714cc954cf541988bf11f3fe91e6e24'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:12.063386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a714cc954cf541988bf11f3fe91e6e24'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:12.109131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a714cc954cf541988bf11f3fe91e6e24'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:12.144598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 900/1806] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:11 2022] DELETE /identity/v3/users/a714cc954cf541988bf11f3fe91e6e24 => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:12.148650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec984946-485e-4157-8cf6-1a8516f5e363 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.149164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec984946-485e-4157-8cf6-1a8516f5e363 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.149164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec984946-485e-4157-8cf6-1a8516f5e363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.186476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ec984946-485e-4157-8cf6-1a8516f5e363 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:12.187726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ec984946-485e-4157-8cf6-1a8516f5e363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.285233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 907/1807] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:12.301745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:12.303738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.390379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d98e4f-574c-45b8-99be-59a4e366ff7e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:12.401162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:12.402277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.402799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.403669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.420997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:12.420997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:12.430043 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 901/1808] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:12.568108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7d117-8535-439a-a1c7-fac990244795 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.568428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7d117-8535-439a-a1c7-fac990244795 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.568712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7d117-8535-439a-a1c7-fac990244795 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.570082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 908/1809] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:12.581806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:12.584228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.595832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dce8f-760a-4468-909b-ec2090fa5d5e None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.596572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dce8f-760a-4468-909b-ec2090fa5d5e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.597264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dce8f-760a-4468-909b-ec2090fa5d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.634543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-323dce8f-760a-4468-909b-ec2090fa5d5e None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:12.635576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-323dce8f-760a-4468-909b-ec2090fa5d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.670317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c223a2-c6ab-4287-8820-5043caf58a25 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:12.683989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:12.685138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.685449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.685719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] PATH_INFO: `/v3/projects/9425cc16384c4f16a92889f135ad0b39` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.687456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9425cc16384c4f16a92889f135ad0b39)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:12.689398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:12.820665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 909/1810] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:12.836806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dce8f-760a-4468-909b-ec2090fa5d5e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:12.838350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-323dce8f-760a-4468-909b-ec2090fa5d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.858074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dce8f-760a-4468-909b-ec2090fa5d5e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:12.868720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826c936-135b-45dc-8c8d-3a0d3ca21cd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:12.868720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c936-135b-45dc-8c8d-3a0d3ca21cd8 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.868720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c936-135b-45dc-8c8d-3a0d3ca21cd8 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.868720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c936-135b-45dc-8c8d-3a0d3ca21cd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.880361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826c936-135b-45dc-8c8d-3a0d3ca21cd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:12.882510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826c936-135b-45dc-8c8d-3a0d3ca21cd8 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:12.885037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9425cc16384c4f16a92889f135ad0b39', 'request_id': 'req-fae789b7-b89a-4de9-9505-f0459a1e915f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:12.900072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9425cc16384c4f16a92889f135ad0b39', 'request_id': 'req-fae789b7-b89a-4de9-9505-f0459a1e915f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:12.914748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 910/1811] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:13.013739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9425cc16384c4f16a92889f135ad0b39', 'request_id': 'req-fae789b7-b89a-4de9-9505-f0459a1e915f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.034306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] The token cache is being invalidate because project 9425cc16384c4f16a92889f135ad0b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:13.034722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.035819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.038259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 902/1812] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:12 2022] DELETE /identity/v3/projects/9425cc16384c4f16a92889f135ad0b39 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:13.042873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826c936-135b-45dc-8c8d-3a0d3ca21cd8 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:13.061175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826c936-135b-45dc-8c8d-3a0d3ca21cd8 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:13.166521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:13.167211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:13.167436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:13.167891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] PATH_INFO: `/v3/users/5a668c884ef94611b148077460bba608` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:13.169475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a668c884ef94611b148077460bba608)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:13.171773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:13.331161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:13.331161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:13.345539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae789b7-b89a-4de9-9505-f0459a1e915f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:13.352112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:13.352951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:13.353304 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:13.353634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] PATH_INFO: `/v3/projects/fb5fb98e61d54e9fbded87e523e1c46e` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:13.354904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb5fb98e61d54e9fbded87e523e1c46e)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:13.357691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:13.373371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:13.423740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a668c884ef94611b148077460bba608'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.493101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a668c884ef94611b148077460bba608'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.494259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a668c884ef94611b148077460bba608'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.509858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb5fb98e61d54e9fbded87e523e1c46e', 'request_id': 'req-c10bfbcd-4f4f-4592-ba53-43241bc221b4'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.519295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a668c884ef94611b148077460bba608'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.522821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb5fb98e61d54e9fbded87e523e1c46e', 'request_id': 'req-c10bfbcd-4f4f-4592-ba53-43241bc221b4'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.529871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a668c884ef94611b148077460bba608'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.542657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 911/1813] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:13 2022] DELETE /identity/v3/users/5a668c884ef94611b148077460bba608 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:13.748520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb5fb98e61d54e9fbded87e523e1c46e', 'request_id': 'req-c10bfbcd-4f4f-4592-ba53-43241bc221b4'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.770322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] The token cache is being invalidate because project fb5fb98e61d54e9fbded87e523e1c46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:13.770875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.772908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.778772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 903/1814] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:13 2022] DELETE /identity/v3/projects/fb5fb98e61d54e9fbded87e523e1c46e => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:13.781129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:13.782763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:13.872698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333855c3-b350-4cd6-ac5d-16d8a6699ac0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:13.879289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:13.879987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:13.880557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:13.880839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] PATH_INFO: `/v3/users/caae89653f244fe7aaad4a8fa0150721` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:13.882050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caae89653f244fe7aaad4a8fa0150721)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:13.884158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:14.101055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caae89653f244fe7aaad4a8fa0150721'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.231803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caae89653f244fe7aaad4a8fa0150721'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.233273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caae89653f244fe7aaad4a8fa0150721'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.286374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caae89653f244fe7aaad4a8fa0150721'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.304727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caae89653f244fe7aaad4a8fa0150721'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.315543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 912/1815] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:13 2022] DELETE /identity/v3/users/caae89653f244fe7aaad4a8fa0150721 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:14.808310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:14.996775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:14.998876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:15.112679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bfbcd-4f4f-4592-ba53-43241bc221b4 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.200049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.200877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.201279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.201686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] PATH_INFO: `/v3/projects/2da1ee3697554d01b14074cc55ffe49b` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.203109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da1ee3697554d01b14074cc55ffe49b)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.210654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.374943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da1ee3697554d01b14074cc55ffe49b', 'request_id': 'req-ccd82789-fe89-4b17-b969-11f05bf31d7b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.386962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da1ee3697554d01b14074cc55ffe49b', 'request_id': 'req-ccd82789-fe89-4b17-b969-11f05bf31d7b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.474895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da1ee3697554d01b14074cc55ffe49b', 'request_id': 'req-ccd82789-fe89-4b17-b969-11f05bf31d7b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.475713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:15.483207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.486298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:15.506269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] The token cache is being invalidate because project 2da1ee3697554d01b14074cc55ffe49b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:15.506269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.506269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ccd82789-fe89-4b17-b969-11f05bf31d7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.512404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba88a06-5b06-4d67-81d8-d1304c8810b9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.524299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 904/1816] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:14 2022] DELETE /identity/v3/projects/2da1ee3697554d01b14074cc55ffe49b => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:15.525650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925532b-2a34-41ab-9bbc-e7a5d04c7acf None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.525936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925532b-2a34-41ab-9bbc-e7a5d04c7acf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.526322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925532b-2a34-41ab-9bbc-e7a5d04c7acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.577102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c925532b-2a34-41ab-9bbc-e7a5d04c7acf None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:15.581002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c925532b-2a34-41ab-9bbc-e7a5d04c7acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:15.632318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.634728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.635376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.636010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] PATH_INFO: `/v3/projects/69ffb76b6f574f1bab8bc5cbd7ac9d95` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.638965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ffb76b6f574f1bab8bc5cbd7ac9d95)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.643131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.691558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 905/1817] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:15.703259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c30e1b-2ead-4332-8b92-d8dd0a60ba83 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.703259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c30e1b-2ead-4332-8b92-d8dd0a60ba83 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.703259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c30e1b-2ead-4332-8b92-d8dd0a60ba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.726420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-81c30e1b-2ead-4332-8b92-d8dd0a60ba83 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:15.727985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-81c30e1b-2ead-4332-8b92-d8dd0a60ba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:15.736816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 906/1818] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:15.742711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c30e1b-2ead-4332-8b92-d8dd0a60ba83 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.757824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c30e1b-2ead-4332-8b92-d8dd0a60ba83 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.763548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.764215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.764503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.764769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.765737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:15.765866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.767424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.784413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 907/1819] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:15.785734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.799525 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c1feb-1066-4a0e-938d-33e1a70a3f72 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.808805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a155cf6-acf1-4531-9d8e-5e97b54cf816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.809759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155cf6-acf1-4531-9d8e-5e97b54cf816 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.809953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155cf6-acf1-4531-9d8e-5e97b54cf816 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.810381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155cf6-acf1-4531-9d8e-5e97b54cf816 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.812172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a155cf6-acf1-4531-9d8e-5e97b54cf816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.813390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a155cf6-acf1-4531-9d8e-5e97b54cf816 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.841368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ffb76b6f574f1bab8bc5cbd7ac9d95', 'request_id': 'req-be7b0297-f820-4a93-91d5-00cb507e3820'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.849038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ffb76b6f574f1bab8bc5cbd7ac9d95', 'request_id': 'req-be7b0297-f820-4a93-91d5-00cb507e3820'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.949104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 908/1820] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:15.955511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a155cf6-acf1-4531-9d8e-5e97b54cf816 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.044447 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a155cf6-acf1-4531-9d8e-5e97b54cf816 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.053367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62787ca3-3602-4366-828f-164b528cd010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.054256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62787ca3-3602-4366-828f-164b528cd010 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.054536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62787ca3-3602-4366-828f-164b528cd010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.054821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62787ca3-3602-4366-828f-164b528cd010 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.055466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62787ca3-3602-4366-828f-164b528cd010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.057671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62787ca3-3602-4366-828f-164b528cd010 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.070596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ffb76b6f574f1bab8bc5cbd7ac9d95', 'request_id': 'req-be7b0297-f820-4a93-91d5-00cb507e3820'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:16.090915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] The token cache is being invalidate because project 69ffb76b6f574f1bab8bc5cbd7ac9d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:16.091506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:16.093387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-be7b0297-f820-4a93-91d5-00cb507e3820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:16.096062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 913/1821] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:15 2022] DELETE /identity/v3/projects/69ffb76b6f574f1bab8bc5cbd7ac9d95 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:16.106902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5df388-64dd-4196-bf34-29a5ceeb092a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.106902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5df388-64dd-4196-bf34-29a5ceeb092a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.106902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5df388-64dd-4196-bf34-29a5ceeb092a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.158096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-df5df388-64dd-4196-bf34-29a5ceeb092a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:16.162793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-df5df388-64dd-4196-bf34-29a5ceeb092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:16.200519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 909/1822] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/users => generated 284 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:16.200519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01d3c-2263-4b75-94f2-2ce7f61c6cd3 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.200519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01d3c-2263-4b75-94f2-2ce7f61c6cd3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.200519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01d3c-2263-4b75-94f2-2ce7f61c6cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.200802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 914/1823] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:16.201932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5df388-64dd-4196-bf34-29a5ceeb092a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.203398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-df5df388-64dd-4196-bf34-29a5ceeb092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:16.234491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5df388-64dd-4196-bf34-29a5ceeb092a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.242269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a045d7e-94f9-4ef3-ad48-97d83b1461d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.243849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a045d7e-94f9-4ef3-ad48-97d83b1461d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.245961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a045d7e-94f9-4ef3-ad48-97d83b1461d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.246495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a045d7e-94f9-4ef3-ad48-97d83b1461d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.249109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a045d7e-94f9-4ef3-ad48-97d83b1461d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.251531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-03d01d3c-2263-4b75-94f2-2ce7f61c6cd3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:16.253244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a045d7e-94f9-4ef3-ad48-97d83b1461d0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.255677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-03d01d3c-2263-4b75-94f2-2ce7f61c6cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:16.267284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 910/1824] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:16.270707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d01d3c-2263-4b75-94f2-2ce7f61c6cd3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.283468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 915/1825] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:16.287704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d01d3c-2263-4b75-94f2-2ce7f61c6cd3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.289565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a045d7e-94f9-4ef3-ad48-97d83b1461d0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.295036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.295813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.296066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.296339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.297092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:16.297617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.299364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.307441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a045d7e-94f9-4ef3-ad48-97d83b1461d0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.318330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 911/1826] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:16.318537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be72f0-8804-462a-affe-8f70f4b11010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.318537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be72f0-8804-462a-affe-8f70f4b11010 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.318537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be72f0-8804-462a-affe-8f70f4b11010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.318537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be72f0-8804-462a-affe-8f70f4b11010 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.318896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be72f0-8804-462a-affe-8f70f4b11010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.320987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be72f0-8804-462a-affe-8f70f4b11010 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.339873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 916/1827] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:16.340087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.343562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be72f0-8804-462a-affe-8f70f4b11010 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.376313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec543b-40cc-4ed9-a0f7-84aba8d14968 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.379951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be72f0-8804-462a-affe-8f70f4b11010 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.387283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd5cea-459e-4f27-9280-f393bf62d661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.395744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7547184-c652-4278-877b-f2cf6b44282e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.395961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd5cea-459e-4f27-9280-f393bf62d661 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.396279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd5cea-459e-4f27-9280-f393bf62d661 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.396826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd5cea-459e-4f27-9280-f393bf62d661 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.397718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd5cea-459e-4f27-9280-f393bf62d661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.398513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7547184-c652-4278-877b-f2cf6b44282e admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.399067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7547184-c652-4278-877b-f2cf6b44282e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.399697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7547184-c652-4278-877b-f2cf6b44282e admin admin] PATH_INFO: `/v3/projects/66c607f3aeee4a978d1e8e25636ce542/users/1e91451aae624aab87b8fee4112c65b2/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.401896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd5cea-459e-4f27-9280-f393bf62d661 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.430264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7547184-c652-4278-877b-f2cf6b44282e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c607f3aeee4a978d1e8e25636ce542, user_id=1e91451aae624aab87b8fee4112c65b2, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.437321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7547184-c652-4278-877b-f2cf6b44282e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.553417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 917/1828] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:16 2022] PUT /identity/v3/projects/66c607f3aeee4a978d1e8e25636ce542/users/1e91451aae624aab87b8fee4112c65b2/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:16.565405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34b897-bbe9-4d28-87e9-6b68e85eddac None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.565958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34b897-bbe9-4d28-87e9-6b68e85eddac None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.566974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34b897-bbe9-4d28-87e9-6b68e85eddac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.580473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 912/1829] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/projects => generated 391 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:16.596745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd5cea-459e-4f27-9280-f393bf62d661 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.615866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-3b34b897-bbe9-4d28-87e9-6b68e85eddac None None] MFA Rules not processed for user `1e91451aae624aab87b8fee4112c65b2`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:16.617294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3b34b897-bbe9-4d28-87e9-6b68e85eddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:16.708621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd5cea-459e-4f27-9280-f393bf62d661 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.714038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ac96c-de72-4dca-b55c-c2095a69f369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.714682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ac96c-de72-4dca-b55c-c2095a69f369 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.715023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ac96c-de72-4dca-b55c-c2095a69f369 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.715442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ac96c-de72-4dca-b55c-c2095a69f369 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.716668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ac96c-de72-4dca-b55c-c2095a69f369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.718816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ac96c-de72-4dca-b55c-c2095a69f369 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.719921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 918/1830] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:16.724167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34b897-bbe9-4d28-87e9-6b68e85eddac None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.740864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34b897-bbe9-4d28-87e9-6b68e85eddac None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.746486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d78541-4f4b-47a3-b3fa-fa139eb2b93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.747795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78541-4f4b-47a3-b3fa-fa139eb2b93b admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.748156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78541-4f4b-47a3-b3fa-fa139eb2b93b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.748500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78541-4f4b-47a3-b3fa-fa139eb2b93b admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.749228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d78541-4f4b-47a3-b3fa-fa139eb2b93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.751312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d78541-4f4b-47a3-b3fa-fa139eb2b93b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.840136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 913/1831] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/users => generated 296 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:16.841928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3d1de-2cd7-4f5f-ae61-434b2d1a901f None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.842362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3d1de-2cd7-4f5f-ae61-434b2d1a901f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.842740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3d1de-2cd7-4f5f-ae61-434b2d1a901f None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.844930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 914/1832] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:54:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:16.846084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3d1de-2cd7-4f5f-ae61-434b2d1a901f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.861186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3d1de-2cd7-4f5f-ae61-434b2d1a901f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.866225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e83ef-309b-41d4-97a0-02cd97bd6cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.866882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e83ef-309b-41d4-97a0-02cd97bd6cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.867164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e83ef-309b-41d4-97a0-02cd97bd6cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.867434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e83ef-309b-41d4-97a0-02cd97bd6cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.868082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e83ef-309b-41d4-97a0-02cd97bd6cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.869863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e83ef-309b-41d4-97a0-02cd97bd6cc0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.882189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 915/1833] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:16.883352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518565-ed53-4b9a-a819-7ab1fd7041d3 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.883651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518565-ed53-4b9a-a819-7ab1fd7041d3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.883930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518565-ed53-4b9a-a819-7ab1fd7041d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.895253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 919/1834] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:16.897576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d78541-4f4b-47a3-b3fa-fa139eb2b93b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.922872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3e518565-ed53-4b9a-a819-7ab1fd7041d3 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:16.924959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3e518565-ed53-4b9a-a819-7ab1fd7041d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:16.979094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d78541-4f4b-47a3-b3fa-fa139eb2b93b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.986487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f6c55-95f1-4a30-b828-793b01f55a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.987340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f6c55-95f1-4a30-b828-793b01f55a8b admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.987744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f6c55-95f1-4a30-b828-793b01f55a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.988378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f6c55-95f1-4a30-b828-793b01f55a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.989051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f6c55-95f1-4a30-b828-793b01f55a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.991299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f6c55-95f1-4a30-b828-793b01f55a8b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.003530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 920/1835] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:17.004001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f6c55-95f1-4a30-b828-793b01f55a8b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.016546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f6c55-95f1-4a30-b828-793b01f55a8b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.022645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.023274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.023555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.023826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.024458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.025988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 916/1836] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:17.026490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.027023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e518565-ed53-4b9a-a819-7ab1fd7041d3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.039649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e518565-ed53-4b9a-a819-7ab1fd7041d3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.044825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399ad92-191a-4c09-b450-85441f4130a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.045513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399ad92-191a-4c09-b450-85441f4130a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.045911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399ad92-191a-4c09-b450-85441f4130a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.046049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399ad92-191a-4c09-b450-85441f4130a2 admin admin] PATH_INFO: `/v3/projects/4bdbe07f152a498f970f07e200999c97/users/b3f5308274ec4bb284aa3d162728a733/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.064082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399ad92-191a-4c09-b450-85441f4130a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bdbe07f152a498f970f07e200999c97, user_id=b3f5308274ec4bb284aa3d162728a733, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.067359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399ad92-191a-4c09-b450-85441f4130a2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.115821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 921/1837] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:17 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:17.121201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.121201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:17.134555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0c131-7a32-41de-b48e-de4d07d0cce2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.141146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a596f82-1a7e-407d-923c-6ae64e59bf99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.141761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a596f82-1a7e-407d-923c-6ae64e59bf99 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.142044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a596f82-1a7e-407d-923c-6ae64e59bf99 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.142367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a596f82-1a7e-407d-923c-6ae64e59bf99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.148117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 917/1838] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:17 2022] PUT /identity/v3/projects/4bdbe07f152a498f970f07e200999c97/users/b3f5308274ec4bb284aa3d162728a733/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:17.148117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399ad92-191a-4c09-b450-85441f4130a2 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.158612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a596f82-1a7e-407d-923c-6ae64e59bf99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.160281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a596f82-1a7e-407d-923c-6ae64e59bf99 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.173289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 922/1839] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:17.183261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9ddc-1558-4106-b68b-614be2aa7cdf None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.183617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9ddc-1558-4106-b68b-614be2aa7cdf None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.183897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9ddc-1558-4106-b68b-614be2aa7cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.231961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-dd8c9ddc-1558-4106-b68b-614be2aa7cdf None None] MFA Rules not processed for user `b3f5308274ec4bb284aa3d162728a733`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:17.233250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-dd8c9ddc-1558-4106-b68b-614be2aa7cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:17.289346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399ad92-191a-4c09-b450-85441f4130a2 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.313825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df2fa-bc3b-4933-8df2-9d56b07dfb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.314606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df2fa-bc3b-4933-8df2-9d56b07dfb2f admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.314899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df2fa-bc3b-4933-8df2-9d56b07dfb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.320070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df2fa-bc3b-4933-8df2-9d56b07dfb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.320070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131df2fa-bc3b-4933-8df2-9d56b07dfb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.329502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131df2fa-bc3b-4933-8df2-9d56b07dfb2f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.357304 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 918/1840] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:17.361717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df2fa-bc3b-4933-8df2-9d56b07dfb2f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.374169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df2fa-bc3b-4933-8df2-9d56b07dfb2f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.392588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:17.396743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320916d-fc90-49c2-88c2-c1d740d6283d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.397671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320916d-fc90-49c2-88c2-c1d740d6283d admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.398131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320916d-fc90-49c2-88c2-c1d740d6283d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.418233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320916d-fc90-49c2-88c2-c1d740d6283d admin admin] PATH_INFO: `/v3/projects/329c5ca16f2f475eb47f8eecc7b81915/users/37b7923f2f454759a13d43bf6948b5bb/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.422116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 923/1841] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:17.425969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9ddc-1558-4106-b68b-614be2aa7cdf None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.442978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9ddc-1558-4106-b68b-614be2aa7cdf None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.449208 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c298f80-7d7f-4b17-a031-3e0c8d6c7d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.450907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298f80-7d7f-4b17-a031-3e0c8d6c7d38 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.454003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298f80-7d7f-4b17-a031-3e0c8d6c7d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.454238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320916d-fc90-49c2-88c2-c1d740d6283d admin admin] RBAC: Authorizing `identity:create_grant(project_id=329c5ca16f2f475eb47f8eecc7b81915, user_id=37b7923f2f454759a13d43bf6948b5bb, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.454354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298f80-7d7f-4b17-a031-3e0c8d6c7d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.454354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c298f80-7d7f-4b17-a031-3e0c8d6c7d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.456052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c298f80-7d7f-4b17-a031-3e0c8d6c7d38 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.458016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320916d-fc90-49c2-88c2-c1d740d6283d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.585665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 919/1842] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:17 2022] PUT /identity/v3/projects/329c5ca16f2f475eb47f8eecc7b81915/users/37b7923f2f454759a13d43bf6948b5bb/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:17.596661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320916d-fc90-49c2-88c2-c1d740d6283d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.621254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 924/1843] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:17 2022] POST /identity/v3/projects => generated 391 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:17.629024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c298f80-7d7f-4b17-a031-3e0c8d6c7d38 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.688052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320916d-fc90-49c2-88c2-c1d740d6283d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.782677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c298f80-7d7f-4b17-a031-3e0c8d6c7d38 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.790125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c4bd9-9628-4258-a956-07a2c66b8566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.791218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4bd9-9628-4258-a956-07a2c66b8566 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.791664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4bd9-9628-4258-a956-07a2c66b8566 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.792138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4bd9-9628-4258-a956-07a2c66b8566 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.793181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c4bd9-9628-4258-a956-07a2c66b8566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.795262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf70c79-9bac-4f5c-a403-14d16db5e682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.795897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf70c79-9bac-4f5c-a403-14d16db5e682 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.796068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c4bd9-9628-4258-a956-07a2c66b8566 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.796172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf70c79-9bac-4f5c-a403-14d16db5e682 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.796460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf70c79-9bac-4f5c-a403-14d16db5e682 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.797091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf70c79-9bac-4f5c-a403-14d16db5e682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.808471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf70c79-9bac-4f5c-a403-14d16db5e682 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.812307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 920/1844] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:17.816976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c4bd9-9628-4258-a956-07a2c66b8566 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.832168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c4bd9-9628-4258-a956-07a2c66b8566 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.841701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5ec7c-e4e9-49b5-9b5c-9e3fa34180af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.843600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5ec7c-e4e9-49b5-9b5c-9e3fa34180af admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.844030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5ec7c-e4e9-49b5-9b5c-9e3fa34180af admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.844449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5ec7c-e4e9-49b5-9b5c-9e3fa34180af admin admin] PATH_INFO: `/v3/projects/329c5ca16f2f475eb47f8eecc7b81915/users/37b7923f2f454759a13d43bf6948b5bb/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.848568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5ec7c-e4e9-49b5-9b5c-9e3fa34180af admin admin] RBAC: Authorizing `identity:create_grant(project_id=329c5ca16f2f475eb47f8eecc7b81915, user_id=37b7923f2f454759a13d43bf6948b5bb, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.852626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5ec7c-e4e9-49b5-9b5c-9e3fa34180af admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.881731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 921/1845] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:17 2022] PUT /identity/v3/projects/329c5ca16f2f475eb47f8eecc7b81915/users/37b7923f2f454759a13d43bf6948b5bb/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:17.889341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4e312-52d4-4080-9353-c659f985dc39 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.889632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4e312-52d4-4080-9353-c659f985dc39 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.889906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4e312-52d4-4080-9353-c659f985dc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.940735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 925/1846] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:17 2022] POST /identity/v3/users => generated 302 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:17.942380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf70c79-9bac-4f5c-a403-14d16db5e682 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.949965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ace4e312-52d4-4080-9353-c659f985dc39 None None] MFA Rules not processed for user `37b7923f2f454759a13d43bf6948b5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:17.953092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ace4e312-52d4-4080-9353-c659f985dc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:17.961400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf70c79-9bac-4f5c-a403-14d16db5e682 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.053643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f82608-bbe1-4672-86cb-3f13601b60b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.055130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f82608-bbe1-4672-86cb-3f13601b60b2 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.055735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f82608-bbe1-4672-86cb-3f13601b60b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.056382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f82608-bbe1-4672-86cb-3f13601b60b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.057527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f82608-bbe1-4672-86cb-3f13601b60b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.061166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f82608-bbe1-4672-86cb-3f13601b60b2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.078635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 926/1847] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:18.082267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f82608-bbe1-4672-86cb-3f13601b60b2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.083389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:18.094104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 922/1848] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:18.095651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f82608-bbe1-4672-86cb-3f13601b60b2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.105192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e28ec-457f-466a-888f-b6903b5bfa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.105957 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e28ec-457f-466a-888f-b6903b5bfa77 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.108198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e28ec-457f-466a-888f-b6903b5bfa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.108198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e28ec-457f-466a-888f-b6903b5bfa77 admin admin] PATH_INFO: `/v3/projects/e19488ee30214f3c972427a42d5c65f8/users/fc1446f1bbaa48468e8729d44f543765/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.119767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4e312-52d4-4080-9353-c659f985dc39 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.119767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ace4e312-52d4-4080-9353-c659f985dc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:18.135360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e28ec-457f-466a-888f-b6903b5bfa77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19488ee30214f3c972427a42d5c65f8, user_id=fc1446f1bbaa48468e8729d44f543765, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.140423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e28ec-457f-466a-888f-b6903b5bfa77 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.191575 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4e312-52d4-4080-9353-c659f985dc39 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.197631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d524b-f41f-4771-be4d-b1fb180fffd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.198615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d524b-f41f-4771-be4d-b1fb180fffd7 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.199097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d524b-f41f-4771-be4d-b1fb180fffd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.199517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d524b-f41f-4771-be4d-b1fb180fffd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.219139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d524b-f41f-4771-be4d-b1fb180fffd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.221046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d524b-f41f-4771-be4d-b1fb180fffd7 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.241562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 923/1849] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:18.260781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 927/1850] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:18 2022] PUT /identity/v3/projects/e19488ee30214f3c972427a42d5c65f8/users/fc1446f1bbaa48468e8729d44f543765/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:18.265433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d524b-f41f-4771-be4d-b1fb180fffd7 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.281974 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d524b-f41f-4771-be4d-b1fb180fffd7 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.397086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477afa6-4317-43f4-9f5a-ff3680148a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.397882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477afa6-4317-43f4-9f5a-ff3680148a28 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.398363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477afa6-4317-43f4-9f5a-ff3680148a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.398782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477afa6-4317-43f4-9f5a-ff3680148a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.399702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477afa6-4317-43f4-9f5a-ff3680148a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.401501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477afa6-4317-43f4-9f5a-ff3680148a28 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.415937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 924/1851] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:18.421174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e28ec-457f-466a-888f-b6903b5bfa77 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.441090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e28ec-457f-466a-888f-b6903b5bfa77 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.446474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd248bf7-3a5f-440a-a9b2-d640e64131a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.447235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248bf7-3a5f-440a-a9b2-d640e64131a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.447669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248bf7-3a5f-440a-a9b2-d640e64131a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.448004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248bf7-3a5f-440a-a9b2-d640e64131a6 admin admin] PATH_INFO: `/v3/projects/e19488ee30214f3c972427a42d5c65f8/users/fc1446f1bbaa48468e8729d44f543765/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.450272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd248bf7-3a5f-440a-a9b2-d640e64131a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19488ee30214f3c972427a42d5c65f8, user_id=fc1446f1bbaa48468e8729d44f543765, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.453280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd248bf7-3a5f-440a-a9b2-d640e64131a6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.465104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 928/1852] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:18 2022] PUT /identity/v3/projects/e19488ee30214f3c972427a42d5c65f8/users/fc1446f1bbaa48468e8729d44f543765/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:18.476781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a6c2-e0f6-4068-88b2-c12b0fa52fb6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.477149 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a6c2-e0f6-4068-88b2-c12b0fa52fb6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.482041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a6c2-e0f6-4068-88b2-c12b0fa52fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.523572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd248bf7-3a5f-440a-a9b2-d640e64131a6 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.540224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3000a6c2-e0f6-4068-88b2-c12b0fa52fb6 None None] MFA Rules not processed for user `fc1446f1bbaa48468e8729d44f543765`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:18.541811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3000a6c2-e0f6-4068-88b2-c12b0fa52fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:18.626511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd248bf7-3a5f-440a-a9b2-d640e64131a6 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.632401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af8d9-e935-4bf8-91a3-60adb5f5aaca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.633056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af8d9-e935-4bf8-91a3-60adb5f5aaca service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.641534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af8d9-e935-4bf8-91a3-60adb5f5aaca service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.641905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af8d9-e935-4bf8-91a3-60adb5f5aaca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.728532 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 925/1853] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:18.750581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0af8d9-e935-4bf8-91a3-60adb5f5aaca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.751314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a6c2-e0f6-4068-88b2-c12b0fa52fb6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.752204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0af8d9-e935-4bf8-91a3-60adb5f5aaca service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.752751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3000a6c2-e0f6-4068-88b2-c12b0fa52fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:18.763073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 929/1854] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:18.769218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a6c2-e0f6-4068-88b2-c12b0fa52fb6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.771702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.772257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.772541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.772820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.784901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.786608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.799653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 926/1855] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:18 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:18.817222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:18.896567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af8d9-e935-4bf8-91a3-60adb5f5aaca service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.906061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af8d9-e935-4bf8-91a3-60adb5f5aaca service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.910013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1cad1-33b4-49bc-8c07-66b4516d4b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.910676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1cad1-33b4-49bc-8c07-66b4516d4b3f service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.910956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1cad1-33b4-49bc-8c07-66b4516d4b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.911228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1cad1-33b4-49bc-8c07-66b4516d4b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.974226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1cad1-33b4-49bc-8c07-66b4516d4b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.975724 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1cad1-33b4-49bc-8c07-66b4516d4b3f service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.986985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 930/1856] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:19.435089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:20.385068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:21.259676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:21.951922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:22.053660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:23.832204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:24.758471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:24.760096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:24.840672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a0ac4-ba65-4181-8d18-e150a101469d service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:24.847350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:24.848013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:24.848300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:24.848576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] PATH_INFO: `/v3/users/63228e006c2047208959b5a27d0a2aae` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:24.849959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63228e006c2047208959b5a27d0a2aae)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:24.851969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:24.885740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:25.052260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63228e006c2047208959b5a27d0a2aae'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.139225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63228e006c2047208959b5a27d0a2aae'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.147169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63228e006c2047208959b5a27d0a2aae'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.148408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63228e006c2047208959b5a27d0a2aae'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.158306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63228e006c2047208959b5a27d0a2aae'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.179947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 927/1857] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:24 2022] DELETE /identity/v3/users/63228e006c2047208959b5a27d0a2aae => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:25.185367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfbdf0-cd10-4f75-bf48-c38868d449dc None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.187076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfbdf0-cd10-4f75-bf48-c38868d449dc None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.187659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfbdf0-cd10-4f75-bf48-c38868d449dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.222226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-57bfbdf0-cd10-4f75-bf48-c38868d449dc None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:25.224209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-57bfbdf0-cd10-4f75-bf48-c38868d449dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:25.286294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 931/1858] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:25.296009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:25.297613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:25.363671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72a77d-6d92-46fa-a082-f7543000de41 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:25.367787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:25.370080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.370080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.370080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.379201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:25.380632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:25.392940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 928/1859] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:26.038272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfbdf0-cd10-4f75-bf48-c38868d449dc None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.039846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-57bfbdf0-cd10-4f75-bf48-c38868d449dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.054714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfbdf0-cd10-4f75-bf48-c38868d449dc None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:26.059514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:26.060204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:26.061242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:26.061526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] PATH_INFO: `/v3/projects/9f46ec1cc9144fbfa5f3db291668ca67` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:26.063044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f46ec1cc9144fbfa5f3db291668ca67)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:26.065141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:26.168513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.169908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.186850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97c9c8-d0de-4b4e-a3a9-4cea1705576a service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:26.193039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:26.193785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:26.200160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:26.200160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] PATH_INFO: `/v3/users/b3f5308274ec4bb284aa3d162728a733` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:26.200160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f5308274ec4bb284aa3d162728a733)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:26.200160 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:26.235854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f46ec1cc9144fbfa5f3db291668ca67', 'request_id': 'req-b2065a77-052d-4714-aed9-d55b64d87e23'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.242318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f46ec1cc9144fbfa5f3db291668ca67', 'request_id': 'req-b2065a77-052d-4714-aed9-d55b64d87e23'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.327598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f46ec1cc9144fbfa5f3db291668ca67', 'request_id': 'req-b2065a77-052d-4714-aed9-d55b64d87e23'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.338462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] The token cache is being invalidate because project 9f46ec1cc9144fbfa5f3db291668ca67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:26.338754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.339395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.341866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 932/1860] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:26 2022] DELETE /identity/v3/projects/9f46ec1cc9144fbfa5f3db291668ca67 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:26.345807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.347164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.412592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f5308274ec4bb284aa3d162728a733'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.421256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2065a77-052d-4714-aed9-d55b64d87e23 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:26.427450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:26.428125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:26.428455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:26.428736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] PATH_INFO: `/v3/users/42db8520a6974a91a9c132217be3af7d` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:26.429928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42db8520a6974a91a9c132217be3af7d)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:26.431984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:26.522423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:26.522423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f5308274ec4bb284aa3d162728a733'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.522423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f5308274ec4bb284aa3d162728a733'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.522423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f5308274ec4bb284aa3d162728a733'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.544878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f134266-23cc-47de-83fd-23775e3995a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f5308274ec4bb284aa3d162728a733'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.578628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 929/1861] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:26 2022] DELETE /identity/v3/users/b3f5308274ec4bb284aa3d162728a733 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:26.579785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aafe1c-9be8-42af-9dbb-ac1a120ed07f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:26.580081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aafe1c-9be8-42af-9dbb-ac1a120ed07f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:26.580359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aafe1c-9be8-42af-9dbb-ac1a120ed07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:26.605806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-61aafe1c-9be8-42af-9dbb-ac1a120ed07f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:26.607742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-61aafe1c-9be8-42af-9dbb-ac1a120ed07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.618393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42db8520a6974a91a9c132217be3af7d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.676157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 930/1862] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:26.686383 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aafe1c-9be8-42af-9dbb-ac1a120ed07f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.688346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-61aafe1c-9be8-42af-9dbb-ac1a120ed07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.753690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aafe1c-9be8-42af-9dbb-ac1a120ed07f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:26.757492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:26.758309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:26.758596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:26.759004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:26.766485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42db8520a6974a91a9c132217be3af7d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.770718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42db8520a6974a91a9c132217be3af7d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.772356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:26.773819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:26.784401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42db8520a6974a91a9c132217be3af7d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.785317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 931/1863] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:26.792478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42db8520a6974a91a9c132217be3af7d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.803292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 933/1864] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:26 2022] DELETE /identity/v3/users/42db8520a6974a91a9c132217be3af7d => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:26.991710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.993129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.006015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670ac47-18e2-4f31-b59b-5083bc527b48 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.101647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.102551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.102836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.103119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] PATH_INFO: `/v3/users/1e91451aae624aab87b8fee4112c65b2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.104465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e91451aae624aab87b8fee4112c65b2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.106511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.152006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:27.296115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.312226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.322077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e91451aae624aab87b8fee4112c65b2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.327805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f9786-6f75-4d01-8f79-c06a5a2ea928 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.340266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.341421 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.341930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.342758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] PATH_INFO: `/v3/projects/4bdbe07f152a498f970f07e200999c97` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.344210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bdbe07f152a498f970f07e200999c97)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.346920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.436633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e91451aae624aab87b8fee4112c65b2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.437267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e91451aae624aab87b8fee4112c65b2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.437910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e91451aae624aab87b8fee4112c65b2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.445611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e91451aae624aab87b8fee4112c65b2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.464909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 932/1865] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:26 2022] DELETE /identity/v3/users/1e91451aae624aab87b8fee4112c65b2 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:27.466258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.468639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.527077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bdbe07f152a498f970f07e200999c97', 'request_id': 'req-1581b695-dad6-41fb-b8da-6fb571ce765e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.534067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bdbe07f152a498f970f07e200999c97', 'request_id': 'req-1581b695-dad6-41fb-b8da-6fb571ce765e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.554006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7233b-3f1d-487f-be92-ced0a8124eda admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.623836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.624485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.624767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.625041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] PATH_INFO: `/v3/projects/744c89f7a00942bf8df77a7670af2046` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.626207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=744c89f7a00942bf8df77a7670af2046)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.628151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.643854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bdbe07f152a498f970f07e200999c97', 'request_id': 'req-1581b695-dad6-41fb-b8da-6fb571ce765e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.665548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] The token cache is being invalidate because project 4bdbe07f152a498f970f07e200999c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:27.665847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.666558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1581b695-dad6-41fb-b8da-6fb571ce765e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.671572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 934/1866] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:27 2022] DELETE /identity/v3/projects/4bdbe07f152a498f970f07e200999c97 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:27.671572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b72fd-3bc9-41ff-ab25-a352a2514b7d None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.671572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b72fd-3bc9-41ff-ab25-a352a2514b7d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.671572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b72fd-3bc9-41ff-ab25-a352a2514b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.691974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-cf0b72fd-3bc9-41ff-ab25-a352a2514b7d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:27.693195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cf0b72fd-3bc9-41ff-ab25-a352a2514b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.701527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 935/1867] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:27.702287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b72fd-3bc9-41ff-ab25-a352a2514b7d None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.703517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cf0b72fd-3bc9-41ff-ab25-a352a2514b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.715927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b72fd-3bc9-41ff-ab25-a352a2514b7d None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.721116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.721754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.722030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.722331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] PATH_INFO: `/v3/users/fc1446f1bbaa48468e8729d44f543765` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.723544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1446f1bbaa48468e8729d44f543765)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.728086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.733760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:27.789362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744c89f7a00942bf8df77a7670af2046', 'request_id': 'req-d3c9c95c-8a53-4593-a74a-1067330ec285'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.796291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744c89f7a00942bf8df77a7670af2046', 'request_id': 'req-d3c9c95c-8a53-4593-a74a-1067330ec285'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.870360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744c89f7a00942bf8df77a7670af2046', 'request_id': 'req-d3c9c95c-8a53-4593-a74a-1067330ec285'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.886510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] The token cache is being invalidate because project 744c89f7a00942bf8df77a7670af2046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:27.887015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.887709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.889756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 933/1868] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:27 2022] DELETE /identity/v3/projects/744c89f7a00942bf8df77a7670af2046 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:27.891305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.892789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.945151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1446f1bbaa48468e8729d44f543765'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.960561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9c95c-8a53-4593-a74a-1067330ec285 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.964395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed332eeb-84bd-47f3-bbb8-dea7df43e360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.965134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed332eeb-84bd-47f3-bbb8-dea7df43e360 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.965403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed332eeb-84bd-47f3-bbb8-dea7df43e360 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.965682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed332eeb-84bd-47f3-bbb8-dea7df43e360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.968608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ed332eeb-84bd-47f3-bbb8-dea7df43e360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.024523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1446f1bbaa48468e8729d44f543765'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.025438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1446f1bbaa48468e8729d44f543765'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.043106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1446f1bbaa48468e8729d44f543765'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.053682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed332eeb-84bd-47f3-bbb8-dea7df43e360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.054198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-524994d3-289d-45e1-a752-18aa8cbab163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1446f1bbaa48468e8729d44f543765'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.055807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed332eeb-84bd-47f3-bbb8-dea7df43e360 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.060535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 936/1869] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:27 2022] DELETE /identity/v3/users/fc1446f1bbaa48468e8729d44f543765 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:28.062020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7dced-a490-417f-b9fd-b6f488b54676 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.067759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7dced-a490-417f-b9fd-b6f488b54676 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.067759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7dced-a490-417f-b9fd-b6f488b54676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.068051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 934/1870] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:28.084529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-fae7dced-a490-417f-b9fd-b6f488b54676 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:28.088713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fae7dced-a490-417f-b9fd-b6f488b54676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.197253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 937/1871] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:28.206536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445d10f-2439-48bb-b96a-fe88a5aa70c6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.206931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445d10f-2439-48bb-b96a-fe88a5aa70c6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.207297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445d10f-2439-48bb-b96a-fe88a5aa70c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.228930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c445d10f-2439-48bb-b96a-fe88a5aa70c6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:28.230395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c445d10f-2439-48bb-b96a-fe88a5aa70c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.239000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 935/1872] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:28.243219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7dced-a490-417f-b9fd-b6f488b54676 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.253941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7dced-a490-417f-b9fd-b6f488b54676 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.258852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.259464 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.259745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.260023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.260675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:28.260978 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.262627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.275231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 938/1873] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:28.280942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445d10f-2439-48bb-b96a-fe88a5aa70c6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.291904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445d10f-2439-48bb-b96a-fe88a5aa70c6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.298309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.299013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.299298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.299573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.300389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.302339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.319697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:28.424316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 936/1874] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:28.428545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.439794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de518f75-be29-4c7e-93f5-ba89594d91ca admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.501379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.504890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.504890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.504890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.504890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.505211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.524426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.526249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.543191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58b8dc-e020-4fb2-bd5c-3b8f68e71b08 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.544940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.545633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.545925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.546217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] PATH_INFO: `/v3/projects/66c607f3aeee4a978d1e8e25636ce542` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.547442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c607f3aeee4a978d1e8e25636ce542)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.549426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.735051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 939/1875] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/users => generated 283 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:28.736008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.736880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c607f3aeee4a978d1e8e25636ce542', 'request_id': 'req-6f358b81-6246-47aa-a0c4-979f7b0b33a8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.738183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.744267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c607f3aeee4a978d1e8e25636ce542', 'request_id': 'req-6f358b81-6246-47aa-a0c4-979f7b0b33a8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.751306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbefca-1ed6-4b83-bf28-4d4bbe2a544a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.810750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.811593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.811941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.812288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] PATH_INFO: `/v3/projects/e19488ee30214f3c972427a42d5c65f8` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.813540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19488ee30214f3c972427a42d5c65f8)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.815704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.830234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c607f3aeee4a978d1e8e25636ce542', 'request_id': 'req-6f358b81-6246-47aa-a0c4-979f7b0b33a8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.844405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] The token cache is being invalidate because project 66c607f3aeee4a978d1e8e25636ce542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:28.844744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.848056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.848056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 937/1876] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:28 2022] DELETE /identity/v3/projects/66c607f3aeee4a978d1e8e25636ce542 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:28.850296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.851796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.865354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f358b81-6246-47aa-a0c4-979f7b0b33a8 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.873175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.873175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.873175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.873175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.873529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.874637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.885582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 938/1877] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:28.886289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.888508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.901515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f9024-2423-4e30-84df-93d2ed33433d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.908685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.909329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.909615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.909892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] PATH_INFO: `/v3/users/37b7923f2f454759a13d43bf6948b5bb` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.911080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37b7923f2f454759a13d43bf6948b5bb)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.913010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.969122 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19488ee30214f3c972427a42d5c65f8', 'request_id': 'req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:28.975646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19488ee30214f3c972427a42d5c65f8', 'request_id': 'req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.070770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19488ee30214f3c972427a42d5c65f8', 'request_id': 'req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.083519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] The token cache is being invalidate because project e19488ee30214f3c972427a42d5c65f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:29.083951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.084752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.086767 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 940/1878] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:28 2022] DELETE /identity/v3/projects/e19488ee30214f3c972427a42d5c65f8 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:29.090817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.092497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.130666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37b7923f2f454759a13d43bf6948b5bb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.173616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48a55a-f49a-42d5-a026-d59f7c9b8ff4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.178993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9ae9c-f7ba-414b-8302-48c57b36ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.179717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9ae9c-f7ba-414b-8302-48c57b36ffb6 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.180088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9ae9c-f7ba-414b-8302-48c57b36ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.180374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9ae9c-f7ba-414b-8302-48c57b36ffb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.185309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9ae9c-f7ba-414b-8302-48c57b36ffb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.185309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9ae9c-f7ba-414b-8302-48c57b36ffb6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.194393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 941/1879] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:29.195194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84fd49-fbc7-47e5-bc13-d27f35685a8f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.195472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84fd49-fbc7-47e5-bc13-d27f35685a8f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.195890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84fd49-fbc7-47e5-bc13-d27f35685a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.207638 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37b7923f2f454759a13d43bf6948b5bb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.208467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37b7923f2f454759a13d43bf6948b5bb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.209259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37b7923f2f454759a13d43bf6948b5bb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.215488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-8b84fd49-fbc7-47e5-bc13-d27f35685a8f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:29.216742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8b84fd49-fbc7-47e5-bc13-d27f35685a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.218688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-be6522ff-a060-4a44-8be9-455191e71406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37b7923f2f454759a13d43bf6948b5bb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.228114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 942/1880] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:29.229829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84fd49-fbc7-47e5-bc13-d27f35685a8f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.237049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8b84fd49-fbc7-47e5-bc13-d27f35685a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.242025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 939/1881] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:28 2022] DELETE /identity/v3/users/37b7923f2f454759a13d43bf6948b5bb => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:29.243368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb8e9-7624-4e37-9bda-972c057a27fd None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.243660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb8e9-7624-4e37-9bda-972c057a27fd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.247235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb8e9-7624-4e37-9bda-972c057a27fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.251353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84fd49-fbc7-47e5-bc13-d27f35685a8f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.265365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-15ecb8e9-7624-4e37-9bda-972c057a27fd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:29.266621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-15ecb8e9-7624-4e37-9bda-972c057a27fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.316235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85942d9-f64a-47e8-b69b-e3aab1286e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.316825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85942d9-f64a-47e8-b69b-e3aab1286e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.317102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85942d9-f64a-47e8-b69b-e3aab1286e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.317382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85942d9-f64a-47e8-b69b-e3aab1286e19 admin admin] PATH_INFO: `/v3/projects/0ed33b6b61ed4d7ca2167695c42d9ba4/users/1f526f3b5020430a8a810a083256798b/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.335006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 940/1882] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:29.338163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85942d9-f64a-47e8-b69b-e3aab1286e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed33b6b61ed4d7ca2167695c42d9ba4, user_id=1f526f3b5020430a8a810a083256798b, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.339159 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb8e9-7624-4e37-9bda-972c057a27fd None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.343368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85942d9-f64a-47e8-b69b-e3aab1286e19 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.352666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb8e9-7624-4e37-9bda-972c057a27fd None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.357924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.358597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.358837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.359107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.359750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:29.360049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.361623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.373809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 941/1883] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:54:29.378184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.387512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f90a0f-0636-40d9-95a5-2aa1812413da admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.392308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0bb026-98d1-4616-9779-eb22408e1aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.392962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bb026-98d1-4616-9779-eb22408e1aea admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.393161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bb026-98d1-4616-9779-eb22408e1aea admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.393437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0bb026-98d1-4616-9779-eb22408e1aea admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.394058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0bb026-98d1-4616-9779-eb22408e1aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.394908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:29.395990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0bb026-98d1-4616-9779-eb22408e1aea admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.441862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 943/1884] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:29 2022] PUT /identity/v3/projects/0ed33b6b61ed4d7ca2167695c42d9ba4/users/1f526f3b5020430a8a810a083256798b/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:29.452200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a884bdb-b21e-4b2c-bc3a-040f0fc1d4b4 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.452481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a884bdb-b21e-4b2c-bc3a-040f0fc1d4b4 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.452753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a884bdb-b21e-4b2c-bc3a-040f0fc1d4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.480573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5a884bdb-b21e-4b2c-bc3a-040f0fc1d4b4 None None] MFA Rules not processed for user `1f526f3b5020430a8a810a083256798b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:29.482396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5a884bdb-b21e-4b2c-bc3a-040f0fc1d4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.515758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 942/1885] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:29 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:29.517305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0bb026-98d1-4616-9779-eb22408e1aea admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.571424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 944/1886] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:29.576379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a884bdb-b21e-4b2c-bc3a-040f0fc1d4b4 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.614211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0bb026-98d1-4616-9779-eb22408e1aea admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.620768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.621328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.621616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.621892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.622603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.625328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.649931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a884bdb-b21e-4b2c-bc3a-040f0fc1d4b4 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.655219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccf763-4fbe-49bc-9822-2e91a58afdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.655831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccf763-4fbe-49bc-9822-2e91a58afdc9 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.656114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccf763-4fbe-49bc-9822-2e91a58afdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.656389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccf763-4fbe-49bc-9822-2e91a58afdc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.657027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccf763-4fbe-49bc-9822-2e91a58afdc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.658977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccf763-4fbe-49bc-9822-2e91a58afdc9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.728603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 943/1887] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:29 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:29.728866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 945/1888] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:29 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:29.728983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.728983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.730813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccf763-4fbe-49bc-9822-2e91a58afdc9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.741754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970e361-14d9-4d43-a657-256b0d41de5d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.786238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:29.809907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccf763-4fbe-49bc-9822-2e91a58afdc9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.815018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb83bf-8b29-4a66-a41f-aeaf6566c9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.815639 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb83bf-8b29-4a66-a41f-aeaf6566c9fb admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.815928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb83bf-8b29-4a66-a41f-aeaf6566c9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.816203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb83bf-8b29-4a66-a41f-aeaf6566c9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.816854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb83bf-8b29-4a66-a41f-aeaf6566c9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.818619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb83bf-8b29-4a66-a41f-aeaf6566c9fb admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.819284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.820054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.820340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.820614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] PATH_INFO: `/v3/projects/329c5ca16f2f475eb47f8eecc7b81915` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.821840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=329c5ca16f2f475eb47f8eecc7b81915)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.824247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.833803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 946/1889] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:29.834511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb83bf-8b29-4a66-a41f-aeaf6566c9fb admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.837546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:29.846224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb83bf-8b29-4a66-a41f-aeaf6566c9fb admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.851390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28ae6-8196-4c01-b1f2-fdaf5845a915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.852141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28ae6-8196-4c01-b1f2-fdaf5845a915 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.852431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28ae6-8196-4c01-b1f2-fdaf5845a915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.852708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28ae6-8196-4c01-b1f2-fdaf5845a915 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.853347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28ae6-8196-4c01-b1f2-fdaf5845a915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.855313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28ae6-8196-4c01-b1f2-fdaf5845a915 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.941734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329c5ca16f2f475eb47f8eecc7b81915', 'request_id': 'req-a7932cb9-936f-4379-bd8a-721e86c47dc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.951969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329c5ca16f2f475eb47f8eecc7b81915', 'request_id': 'req-a7932cb9-936f-4379-bd8a-721e86c47dc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:29.963433 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 947/1890] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:29 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:29.964416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28ae6-8196-4c01-b1f2-fdaf5845a915 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.975657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28ae6-8196-4c01-b1f2-fdaf5845a915 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.024632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329c5ca16f2f475eb47f8eecc7b81915', 'request_id': 'req-a7932cb9-936f-4379-bd8a-721e86c47dc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:30.038855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] The token cache is being invalidate because project 329c5ca16f2f475eb47f8eecc7b81915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:30.038855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:30.039327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:30.040934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 944/1891] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:29 2022] DELETE /identity/v3/projects/329c5ca16f2f475eb47f8eecc7b81915 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:30.044640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.044766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ede023-54e7-4a92-8bdf-70e9ac7a27a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.044888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.044977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ede023-54e7-4a92-8bdf-70e9ac7a27a4 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.044977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ede023-54e7-4a92-8bdf-70e9ac7a27a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.044977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ede023-54e7-4a92-8bdf-70e9ac7a27a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.044977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ede023-54e7-4a92-8bdf-70e9ac7a27a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.046421 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ede023-54e7-4a92-8bdf-70e9ac7a27a4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.057691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 948/1892] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:30.058093 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7932cb9-936f-4379-bd8a-721e86c47dc0 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.059600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb6ac8-9374-4777-91ea-04e0a26844b2 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.059745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb6ac8-9374-4777-91ea-04e0a26844b2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.060209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb6ac8-9374-4777-91ea-04e0a26844b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.065436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.066154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.066467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.066679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.067332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.068924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.083678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 945/1893] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:30.083678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.085220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-03bb6ac8-9374-4777-91ea-04e0a26844b2 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:30.085344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.087018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-03bb6ac8-9374-4777-91ea-04e0a26844b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.096239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 949/1894] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:30.097467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb6ac8-9374-4777-91ea-04e0a26844b2 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.105668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e296cee-b67e-43e7-9288-d7d36f63239d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.108168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb6ac8-9374-4777-91ea-04e0a26844b2 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.111707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f70349-e660-43c4-bf2f-7a0d37b6d04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.112386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f70349-e660-43c4-bf2f-7a0d37b6d04f admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.112686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f70349-e660-43c4-bf2f-7a0d37b6d04f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.112966 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f70349-e660-43c4-bf2f-7a0d37b6d04f admin admin] PATH_INFO: `/v3/projects/3bd68644b333428587edac6ed99bdac3/users/686f90851d044d11b3666f00e7bac9d8/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.114256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1054280-4662-4c44-a419-ca26d06c84eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.114990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1054280-4662-4c44-a419-ca26d06c84eb admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.115543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1054280-4662-4c44-a419-ca26d06c84eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.118416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1054280-4662-4c44-a419-ca26d06c84eb admin admin] PATH_INFO: `/v3/projects/0aeb951ef3c04526b8a3d860edfed03a/users/766c4bad8ed04813887811a2ea526cbf/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.130710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f70349-e660-43c4-bf2f-7a0d37b6d04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd68644b333428587edac6ed99bdac3, user_id=686f90851d044d11b3666f00e7bac9d8, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.132260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f70349-e660-43c4-bf2f-7a0d37b6d04f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.139742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1054280-4662-4c44-a419-ca26d06c84eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aeb951ef3c04526b8a3d860edfed03a, user_id=766c4bad8ed04813887811a2ea526cbf, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.142900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1054280-4662-4c44-a419-ca26d06c84eb admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.207768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 946/1895] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:30 2022] PUT /identity/v3/projects/3bd68644b333428587edac6ed99bdac3/users/686f90851d044d11b3666f00e7bac9d8/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:30.211090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5ac6-0283-4f38-8a88-11db5d155490 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.211090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5ac6-0283-4f38-8a88-11db5d155490 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.211090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5ac6-0283-4f38-8a88-11db5d155490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.231456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-811b5ac6-0283-4f38-8a88-11db5d155490 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:30.234655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 950/1896] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:30 2022] PUT /identity/v3/projects/0aeb951ef3c04526b8a3d860edfed03a/users/766c4bad8ed04813887811a2ea526cbf/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:30.235125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-811b5ac6-0283-4f38-8a88-11db5d155490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.235238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581378e-7280-49aa-a571-01789bc8f50a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.235238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581378e-7280-49aa-a571-01789bc8f50a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.235238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581378e-7280-49aa-a571-01789bc8f50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.271422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1581378e-7280-49aa-a571-01789bc8f50a None None] MFA Rules not processed for user `686f90851d044d11b3666f00e7bac9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:30.272658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1581378e-7280-49aa-a571-01789bc8f50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.304215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 947/1897] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:30.305211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b5ac6-0283-4f38-8a88-11db5d155490 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.316067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b5ac6-0283-4f38-8a88-11db5d155490 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.321413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bc638-81b2-4440-9caf-0abfb2a71310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.322126 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bc638-81b2-4440-9caf-0abfb2a71310 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.322546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bc638-81b2-4440-9caf-0abfb2a71310 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.327268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bc638-81b2-4440-9caf-0abfb2a71310 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.329331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02bc638-81b2-4440-9caf-0abfb2a71310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.337018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02bc638-81b2-4440-9caf-0abfb2a71310 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.351816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 948/1898] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:30.355402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bc638-81b2-4440-9caf-0abfb2a71310 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.356359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 951/1899] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:30.363675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581378e-7280-49aa-a571-01789bc8f50a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.373320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bc638-81b2-4440-9caf-0abfb2a71310 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.378526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581378e-7280-49aa-a571-01789bc8f50a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.378682 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.379260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.379542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.379816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.380474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:30.380786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.382513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.383387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63c8d5-9338-476e-b0bc-e6dbcada8840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.384349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63c8d5-9338-476e-b0bc-e6dbcada8840 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.384633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63c8d5-9338-476e-b0bc-e6dbcada8840 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.384901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63c8d5-9338-476e-b0bc-e6dbcada8840 admin admin] PATH_INFO: `/v3/projects/0aeb951ef3c04526b8a3d860edfed03a/users/766c4bad8ed04813887811a2ea526cbf/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.386909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63c8d5-9338-476e-b0bc-e6dbcada8840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aeb951ef3c04526b8a3d860edfed03a, user_id=766c4bad8ed04813887811a2ea526cbf, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.390462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63c8d5-9338-476e-b0bc-e6dbcada8840 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.396445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 949/1900] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:30.397487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.400976 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.401719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 952/1901] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:30 2022] PUT /identity/v3/projects/0aeb951ef3c04526b8a3d860edfed03a/users/766c4bad8ed04813887811a2ea526cbf/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:30.402572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63c8d5-9338-476e-b0bc-e6dbcada8840 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.469852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed5b9d-ebae-4d8e-aed4-657deba5deab admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.478467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63c8d5-9338-476e-b0bc-e6dbcada8840 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.483400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a11f0c-bb31-4f34-805f-6c92ccb8d5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.484011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a11f0c-bb31-4f34-805f-6c92ccb8d5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.484348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a11f0c-bb31-4f34-805f-6c92ccb8d5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.484700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a11f0c-bb31-4f34-805f-6c92ccb8d5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.485341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a11f0c-bb31-4f34-805f-6c92ccb8d5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.487283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a11f0c-bb31-4f34-805f-6c92ccb8d5e7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.520563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d59d22-31cb-4ea1-b242-ca451348f621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.521331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59d22-31cb-4ea1-b242-ca451348f621 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.521615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59d22-31cb-4ea1-b242-ca451348f621 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.521888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59d22-31cb-4ea1-b242-ca451348f621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.578493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d59d22-31cb-4ea1-b242-ca451348f621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.579950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d59d22-31cb-4ea1-b242-ca451348f621 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.589280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 950/1902] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:30.590253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd904eb-7a31-4491-8a2b-5f9da07c5d56 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.590544 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd904eb-7a31-4491-8a2b-5f9da07c5d56 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.590818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd904eb-7a31-4491-8a2b-5f9da07c5d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.596089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 953/1903] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:30 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:30.601096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a11f0c-bb31-4f34-805f-6c92ccb8d5e7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.620316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-fdd904eb-7a31-4491-8a2b-5f9da07c5d56 None None] MFA Rules not processed for user `766c4bad8ed04813887811a2ea526cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:30.621612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fdd904eb-7a31-4491-8a2b-5f9da07c5d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.673705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a11f0c-bb31-4f34-805f-6c92ccb8d5e7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.678912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.679528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.679809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.680214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.680889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.682853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.702966 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 951/1904] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:30.704348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd904eb-7a31-4491-8a2b-5f9da07c5d56 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.716805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd904eb-7a31-4491-8a2b-5f9da07c5d56 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.721664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.722320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.722604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.722883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] PATH_INFO: `/v3/users/686f90851d044d11b3666f00e7bac9d8` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.724012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=686f90851d044d11b3666f00e7bac9d8)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.726067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.768217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 954/1905] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:30 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:30.769052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.770630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.828954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3ca7f-1b1e-43cd-b2c2-1cae38213210 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.832665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6957dd0-addb-4163-ae44-44c78850bf60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.833280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6957dd0-addb-4163-ae44-44c78850bf60 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.833557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6957dd0-addb-4163-ae44-44c78850bf60 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.833826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6957dd0-addb-4163-ae44-44c78850bf60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.843142 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6957dd0-addb-4163-ae44-44c78850bf60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.844591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6957dd0-addb-4163-ae44-44c78850bf60 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.854454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 955/1906] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:30.855323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6957dd0-addb-4163-ae44-44c78850bf60 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.863131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:30.867266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6957dd0-addb-4163-ae44-44c78850bf60 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.873011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01244dd-aa97-43d5-aadd-f742d991e5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.873621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01244dd-aa97-43d5-aadd-f742d991e5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.873905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01244dd-aa97-43d5-aadd-f742d991e5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.874208 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01244dd-aa97-43d5-aadd-f742d991e5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.874904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01244dd-aa97-43d5-aadd-f742d991e5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.876570 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01244dd-aa97-43d5-aadd-f742d991e5e2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.886709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 956/1907] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:30.891011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01244dd-aa97-43d5-aadd-f742d991e5e2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.900523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01244dd-aa97-43d5-aadd-f742d991e5e2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.905263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18053e88-2f9d-429d-b094-54e600e217a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.905850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18053e88-2f9d-429d-b094-54e600e217a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.906156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18053e88-2f9d-429d-b094-54e600e217a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.906435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18053e88-2f9d-429d-b094-54e600e217a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.907067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18053e88-2f9d-429d-b094-54e600e217a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.908631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18053e88-2f9d-429d-b094-54e600e217a0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.918344 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 957/1908] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:54:30.923203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18053e88-2f9d-429d-b094-54e600e217a0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.932722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18053e88-2f9d-429d-b094-54e600e217a0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.936101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '686f90851d044d11b3666f00e7bac9d8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:30.939164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a7eb5-ee6d-4cd9-a540-c5ce9c38eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.939826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a7eb5-ee6d-4cd9-a540-c5ce9c38eae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.940099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a7eb5-ee6d-4cd9-a540-c5ce9c38eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.940372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a7eb5-ee6d-4cd9-a540-c5ce9c38eae5 admin admin] PATH_INFO: `/v3/projects/5117d8b2c5284d249b6e2e57f537c10e/users/0ef8b5e20b824fc88928af9909570a07/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.954944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a7eb5-ee6d-4cd9-a540-c5ce9c38eae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5117d8b2c5284d249b6e2e57f537c10e, user_id=0ef8b5e20b824fc88928af9909570a07, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.956308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:30.957568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a7eb5-ee6d-4cd9-a540-c5ce9c38eae5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.018907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '686f90851d044d11b3666f00e7bac9d8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:31.019535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '686f90851d044d11b3666f00e7bac9d8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:31.020151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '686f90851d044d11b3666f00e7bac9d8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:31.028272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2bff2289-93fd-4497-b305-b84a3905eb4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '686f90851d044d11b3666f00e7bac9d8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:31.047194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 952/1909] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:30 2022] DELETE /identity/v3/users/686f90851d044d11b3666f00e7bac9d8 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:31.051433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c9816-c2da-44e7-add1-26a1fb18a64d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.051813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c9816-c2da-44e7-add1-26a1fb18a64d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.052054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c9816-c2da-44e7-add1-26a1fb18a64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.077063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-788c9816-c2da-44e7-add1-26a1fb18a64d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:31.078337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-788c9816-c2da-44e7-add1-26a1fb18a64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:31.088844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 958/1910] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:30 2022] PUT /identity/v3/projects/5117d8b2c5284d249b6e2e57f537c10e/users/0ef8b5e20b824fc88928af9909570a07/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:31.107746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07caef-a459-4403-94e3-247b389da9ee None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.108057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07caef-a459-4403-94e3-247b389da9ee None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.108288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07caef-a459-4403-94e3-247b389da9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.146461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-7d07caef-a459-4403-94e3-247b389da9ee None None] MFA Rules not processed for user `0ef8b5e20b824fc88928af9909570a07`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:31.146461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7d07caef-a459-4403-94e3-247b389da9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:31.177717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 953/1911] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:31.179591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c9816-c2da-44e7-add1-26a1fb18a64d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.181184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-788c9816-c2da-44e7-add1-26a1fb18a64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:31.217294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 959/1912] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:31.218271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07caef-a459-4403-94e3-247b389da9ee None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.219650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7d07caef-a459-4403-94e3-247b389da9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:31.267237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c9816-c2da-44e7-add1-26a1fb18a64d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.307066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07caef-a459-4403-94e3-247b389da9ee None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.307066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:31.307763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.307763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.308268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.332742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:31.333530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.333930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.334357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.336244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:31.380622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:31.382452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.393131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 960/1913] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:31.395276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.397793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:31.405265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:31.406860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.415850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df1789-5009-4db6-926c-a157d6f43899 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.417706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 954/1914] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:31.421733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ced0-497a-4687-a0d5-8006289f1983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:31.422735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ced0-497a-4687-a0d5-8006289f1983 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.423082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ced0-497a-4687-a0d5-8006289f1983 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.424157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ced0-497a-4687-a0d5-8006289f1983 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.425343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ced0-497a-4687-a0d5-8006289f1983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:31.427935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ced0-497a-4687-a0d5-8006289f1983 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.454413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:31.526744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 961/1915] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:31 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:31.529176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.544604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8791fcf-f78b-480d-a525-adb418780153 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.606448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1a7d7-3ee3-44e3-87b5-037005a0d5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:31.607092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1a7d7-3ee3-44e3-87b5-037005a0d5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.607377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1a7d7-3ee3-44e3-87b5-037005a0d5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.607651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1a7d7-3ee3-44e3-87b5-037005a0d5d1 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.608330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1a7d7-3ee3-44e3-87b5-037005a0d5d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:31.610492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1a7d7-3ee3-44e3-87b5-037005a0d5d1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.746203 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 955/1916] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:31 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:31.751042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ced0-497a-4687-a0d5-8006289f1983 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.761595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ced0-497a-4687-a0d5-8006289f1983 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.766612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:31.767996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.769667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.769667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.769667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:31.770589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.781774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 962/1917] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:31.786165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1a7d7-3ee3-44e3-87b5-037005a0d5d1 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.796231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1a7d7-3ee3-44e3-87b5-037005a0d5d1 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.800914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830fc81-bbd6-4aed-b2d2-ea8f36d20a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:31.801627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830fc81-bbd6-4aed-b2d2-ea8f36d20a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.801883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830fc81-bbd6-4aed-b2d2-ea8f36d20a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.802186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830fc81-bbd6-4aed-b2d2-ea8f36d20a44 admin admin] PATH_INFO: `/v3/projects/8e63a1a71e0d412b96da76f1eb476fee/users/7f0539bdb7fa4635b36f22e3a5fc13ec/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.817546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830fc81-bbd6-4aed-b2d2-ea8f36d20a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e63a1a71e0d412b96da76f1eb476fee, user_id=7f0539bdb7fa4635b36f22e3a5fc13ec, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:31.820211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830fc81-bbd6-4aed-b2d2-ea8f36d20a44 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.824726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.827150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:31.840725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050e766-e3c6-474f-8dd3-14f93d2bd945 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.846943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:31.854061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.854061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.854061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] PATH_INFO: `/v3/projects/3bd68644b333428587edac6ed99bdac3` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.854061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:31.854754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd68644b333428587edac6ed99bdac3)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:31.857503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.908729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 956/1918] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:31 2022] PUT /identity/v3/projects/8e63a1a71e0d412b96da76f1eb476fee/users/7f0539bdb7fa4635b36f22e3a5fc13ec/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:31.910491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830fc81-bbd6-4aed-b2d2-ea8f36d20a44 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.975097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830fc81-bbd6-4aed-b2d2-ea8f36d20a44 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.975697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd68644b333428587edac6ed99bdac3', 'request_id': 'req-ec088cd4-456c-4506-8b6f-df7b15b4539e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:31.982152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd68644b333428587edac6ed99bdac3', 'request_id': 'req-ec088cd4-456c-4506-8b6f-df7b15b4539e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:32.031626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad29f8-44ed-4075-b87a-e59c4f3db5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.032250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad29f8-44ed-4075-b87a-e59c4f3db5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.032533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad29f8-44ed-4075-b87a-e59c4f3db5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.032810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad29f8-44ed-4075-b87a-e59c4f3db5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.033454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad29f8-44ed-4075-b87a-e59c4f3db5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.035123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad29f8-44ed-4075-b87a-e59c4f3db5f2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.044845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 957/1919] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:32.049245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad29f8-44ed-4075-b87a-e59c4f3db5f2 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.057357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd68644b333428587edac6ed99bdac3', 'request_id': 'req-ec088cd4-456c-4506-8b6f-df7b15b4539e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:32.059405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad29f8-44ed-4075-b87a-e59c4f3db5f2 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.064710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69661f4-5752-4d4c-8674-009567d40b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.065391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69661f4-5752-4d4c-8674-009567d40b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.065673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69661f4-5752-4d4c-8674-009567d40b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.065971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69661f4-5752-4d4c-8674-009567d40b31 admin admin] PATH_INFO: `/v3/projects/8e63a1a71e0d412b96da76f1eb476fee/users/7f0539bdb7fa4635b36f22e3a5fc13ec/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.067988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69661f4-5752-4d4c-8674-009567d40b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e63a1a71e0d412b96da76f1eb476fee, user_id=7f0539bdb7fa4635b36f22e3a5fc13ec, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.069856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] The token cache is being invalidate because project 3bd68644b333428587edac6ed99bdac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:32.070182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:32.070768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69661f4-5752-4d4c-8674-009567d40b31 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.070899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ec088cd4-456c-4506-8b6f-df7b15b4539e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:32.072600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 963/1920] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:31 2022] DELETE /identity/v3/projects/3bd68644b333428587edac6ed99bdac3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:32.078290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe8872-cba1-4a1d-99b7-2ab3f64011ca None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.078638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe8872-cba1-4a1d-99b7-2ab3f64011ca None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.078844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe8872-cba1-4a1d-99b7-2ab3f64011ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.090390 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 958/1921] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:32 2022] PUT /identity/v3/projects/8e63a1a71e0d412b96da76f1eb476fee/users/7f0539bdb7fa4635b36f22e3a5fc13ec/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:32.098817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a64a5-4432-428b-bf5b-6cde4cfa47db None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.099104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a64a5-4432-428b-bf5b-6cde4cfa47db None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.099383 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a64a5-4432-428b-bf5b-6cde4cfa47db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.102356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-9fbe8872-cba1-4a1d-99b7-2ab3f64011ca None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:32.103588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9fbe8872-cba1-4a1d-99b7-2ab3f64011ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:32.133307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-366a64a5-4432-428b-bf5b-6cde4cfa47db None None] MFA Rules not processed for user `7f0539bdb7fa4635b36f22e3a5fc13ec`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:32.134465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-366a64a5-4432-428b-bf5b-6cde4cfa47db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:32.196473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 964/1922] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:32.204155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1dfbd-5f96-4475-821d-4e0add3e1541 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.204445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1dfbd-5f96-4475-821d-4e0add3e1541 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.204759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1dfbd-5f96-4475-821d-4e0add3e1541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.224315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 959/1923] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:32.228119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-b8b1dfbd-5f96-4475-821d-4e0add3e1541 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:32.228119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b8b1dfbd-5f96-4475-821d-4e0add3e1541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:32.237430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a64a5-4432-428b-bf5b-6cde4cfa47db None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.239630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-366a64a5-4432-428b-bf5b-6cde4cfa47db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:32.242214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 965/1924] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:32.243051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1dfbd-5f96-4475-821d-4e0add3e1541 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.253089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1dfbd-5f96-4475-821d-4e0add3e1541 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.259210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.260214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.260494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.260770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.261429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:32.261993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.263742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.284366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 966/1925] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:32.288668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.299579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93447cf3-b258-42a3-958e-dcaa0b026c24 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.307530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ab88b-e161-4f1d-b472-7293b608fca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.308231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ab88b-e161-4f1d-b472-7293b608fca1 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.309026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ab88b-e161-4f1d-b472-7293b608fca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.309413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ab88b-e161-4f1d-b472-7293b608fca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.310655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ab88b-e161-4f1d-b472-7293b608fca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.314100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ab88b-e161-4f1d-b472-7293b608fca1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.325984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a64a5-4432-428b-bf5b-6cde4cfa47db None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.337817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b491-035c-4b20-96ef-880cd84a29ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.338214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b491-035c-4b20-96ef-880cd84a29ae service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.338550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b491-035c-4b20-96ef-880cd84a29ae service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.338876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06b491-035c-4b20-96ef-880cd84a29ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.359283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06b491-035c-4b20-96ef-880cd84a29ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.361170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06b491-035c-4b20-96ef-880cd84a29ae service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.374359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 960/1926] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:32.431152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 967/1927] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:32 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:32.436114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b491-035c-4b20-96ef-880cd84a29ae service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.453509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06b491-035c-4b20-96ef-880cd84a29ae service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.515981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c537120-51ee-4518-8e95-e76b6148b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.516605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c537120-51ee-4518-8e95-e76b6148b77b admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.516938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c537120-51ee-4518-8e95-e76b6148b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.517225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c537120-51ee-4518-8e95-e76b6148b77b admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.518197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c537120-51ee-4518-8e95-e76b6148b77b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.520253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c537120-51ee-4518-8e95-e76b6148b77b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.541380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:32.648659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 961/1928] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:32 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:32.661152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ab88b-e161-4f1d-b472-7293b608fca1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.678107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ab88b-e161-4f1d-b472-7293b608fca1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.684284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecc45c-d681-4179-aa1e-6dd3affc55e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.684893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecc45c-d681-4179-aa1e-6dd3affc55e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.686986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecc45c-d681-4179-aa1e-6dd3affc55e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.686986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecc45c-d681-4179-aa1e-6dd3affc55e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.691021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecc45c-d681-4179-aa1e-6dd3affc55e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.692693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecc45c-d681-4179-aa1e-6dd3affc55e4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.703530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 968/1929] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:32.713102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c537120-51ee-4518-8e95-e76b6148b77b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.724677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c537120-51ee-4518-8e95-e76b6148b77b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.734299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065027d-38f8-48d0-8e28-19f8190b2d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.734299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065027d-38f8-48d0-8e28-19f8190b2d82 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.734299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065027d-38f8-48d0-8e28-19f8190b2d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.734299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065027d-38f8-48d0-8e28-19f8190b2d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.735043 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065027d-38f8-48d0-8e28-19f8190b2d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.739081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065027d-38f8-48d0-8e28-19f8190b2d82 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.757014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 962/1930] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:32.763712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecc45c-d681-4179-aa1e-6dd3affc55e4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.775550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecc45c-d681-4179-aa1e-6dd3affc55e4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.780951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.781634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.781917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.782222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] PATH_INFO: `/v3/projects/e823ff45e4e94c5abb5a40123d9a224a/users/789e22058cff40ec86ad10689d060abc/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.799863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e823ff45e4e94c5abb5a40123d9a224a, user_id=789e22058cff40ec86ad10689d060abc, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.802612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.882765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 969/1931] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:32 2022] PUT /identity/v3/projects/e823ff45e4e94c5abb5a40123d9a224a/users/789e22058cff40ec86ad10689d060abc/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:32.895236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0e3fc-9d7e-4b60-a318-e68573f01a09 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.895436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0e3fc-9d7e-4b60-a318-e68573f01a09 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.895712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0e3fc-9d7e-4b60-a318-e68573f01a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.929812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6bf0e3fc-9d7e-4b60-a318-e68573f01a09 None None] MFA Rules not processed for user `789e22058cff40ec86ad10689d060abc`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:32.931075 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6bf0e3fc-9d7e-4b60-a318-e68573f01a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:32.945415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:33.006720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 963/1932] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:33.130065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:33.131817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:33.192343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a637a3-6230-4abb-8463-f7cffe059282 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:33.195917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:33.196516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:33.196784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:33.197058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:33.205758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:33.207227 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:33.217037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 970/1933] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:33.238837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0e3fc-9d7e-4b60-a318-e68573f01a09 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:33.241325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:33.243780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:33.303473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0e3fc-9d7e-4b60-a318-e68573f01a09 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:33.308258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:33.308896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:33.309170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:33.309419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b374bbc-9e1a-4546-8477-d0780e1de2e0 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:33.310432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] PATH_INFO: `/v3/users/789e22058cff40ec86ad10689d060abc` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:33.311124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=789e22058cff40ec86ad10689d060abc)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:33.313063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:33.315331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:33.315965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:33.316247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:33.316524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] PATH_INFO: `/v3/users/0ef8b5e20b824fc88928af9909570a07` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:33.317815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef8b5e20b824fc88928af9909570a07)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:33.319861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:33.469072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:33.536659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789e22058cff40ec86ad10689d060abc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.615637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef8b5e20b824fc88928af9909570a07'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.631483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789e22058cff40ec86ad10689d060abc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.632345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789e22058cff40ec86ad10689d060abc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.633602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789e22058cff40ec86ad10689d060abc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.649735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dba28d8f-f724-4831-9c13-78f85cd88d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789e22058cff40ec86ad10689d060abc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.677918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 964/1934] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:33 2022] DELETE /identity/v3/users/789e22058cff40ec86ad10689d060abc => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:33.682298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff02f2-8a7e-4ced-9fa4-8f5897d8b65d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:33.682905 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff02f2-8a7e-4ced-9fa4-8f5897d8b65d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:33.683191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff02f2-8a7e-4ced-9fa4-8f5897d8b65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:33.705237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-1fff02f2-8a7e-4ced-9fa4-8f5897d8b65d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:33.706588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1fff02f2-8a7e-4ced-9fa4-8f5897d8b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:33.720415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef8b5e20b824fc88928af9909570a07'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.721063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef8b5e20b824fc88928af9909570a07'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.756191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef8b5e20b824fc88928af9909570a07'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.771798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-672d4383-cbb9-40e9-87dc-9a46c90dfe94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef8b5e20b824fc88928af9909570a07'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:33.782329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 971/1935] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:33 2022] DELETE /identity/v3/users/0ef8b5e20b824fc88928af9909570a07 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:33.784144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99484621-fc5e-47d8-9494-3075d2137272 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:33.784608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99484621-fc5e-47d8-9494-3075d2137272 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:33.785021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99484621-fc5e-47d8-9494-3075d2137272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:33.786630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 965/1936] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:33.807460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff02f2-8a7e-4ced-9fa4-8f5897d8b65d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:33.811546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1fff02f2-8a7e-4ced-9fa4-8f5897d8b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:33.815563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-99484621-fc5e-47d8-9494-3075d2137272 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:33.817085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-99484621-fc5e-47d8-9494-3075d2137272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:33.861082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:33.898327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 972/1937] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:33.907807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99484621-fc5e-47d8-9494-3075d2137272 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:33.913345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff02f2-8a7e-4ced-9fa4-8f5897d8b65d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:33.973103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:33.973851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:33.974222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:33.974562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:33.987759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:33.989401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:34.001453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 966/1938] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:34.005242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99484621-fc5e-47d8-9494-3075d2137272 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:34.009454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:34.010066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:34.010066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:34.010377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:34.021892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:34.022409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:34.040362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 973/1939] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:34.146123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:34.597276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:34.598814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:34.614093 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781dbc8-791f-47c8-a282-147312d7fa32 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:34.619096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:34.619733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:34.620011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:34.620284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] PATH_INFO: `/v3/projects/e823ff45e4e94c5abb5a40123d9a224a` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:34.621739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e823ff45e4e94c5abb5a40123d9a224a)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:34.624281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:34.628605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:34.641975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:34.643963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:34.666986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e2e05-6176-4129-bb2c-29d60456dbd5 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:34.679942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:34.680704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:34.680986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:34.681266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] PATH_INFO: `/v3/projects/5117d8b2c5284d249b6e2e57f537c10e` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:34.682461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5117d8b2c5284d249b6e2e57f537c10e)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:34.684381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:34.769625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e823ff45e4e94c5abb5a40123d9a224a', 'request_id': 'req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.776189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e823ff45e4e94c5abb5a40123d9a224a', 'request_id': 'req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.799404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5117d8b2c5284d249b6e2e57f537c10e', 'request_id': 'req-d3b6b366-f228-40d4-b455-8dc86a49273f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.806608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5117d8b2c5284d249b6e2e57f537c10e', 'request_id': 'req-d3b6b366-f228-40d4-b455-8dc86a49273f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.886356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e823ff45e4e94c5abb5a40123d9a224a', 'request_id': 'req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.900269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] The token cache is being invalidate because project e823ff45e4e94c5abb5a40123d9a224a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:34.900574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.901252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-bca7d8f2-06f1-402c-9bcb-c72b7c74bb92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.907525 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 967/1940] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:34 2022] DELETE /identity/v3/projects/e823ff45e4e94c5abb5a40123d9a224a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:34.914766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec97f-3786-4294-a0de-05069363c892 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:34.915065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec97f-3786-4294-a0de-05069363c892 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:34.915631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec97f-3786-4294-a0de-05069363c892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:34.934015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5117d8b2c5284d249b6e2e57f537c10e', 'request_id': 'req-d3b6b366-f228-40d4-b455-8dc86a49273f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.953487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] The token cache is being invalidate because project 5117d8b2c5284d249b6e2e57f537c10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:34.953817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.954576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:34.956658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 974/1941] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:34 2022] DELETE /identity/v3/projects/5117d8b2c5284d249b6e2e57f537c10e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:34.960522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:34.962034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:34.973798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-872ec97f-3786-4294-a0de-05069363c892 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:34.975060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-872ec97f-3786-4294-a0de-05069363c892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:35.044971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6b366-f228-40d4-b455-8dc86a49273f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.056256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 968/1942] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:35.056662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.057449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.057750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.058025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] PATH_INFO: `/v3/users/7f0539bdb7fa4635b36f22e3a5fc13ec` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.059385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0539bdb7fa4635b36f22e3a5fc13ec)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.061991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.064853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d417216-00df-42e1-91ae-263148f0d9e3 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.065209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d417216-00df-42e1-91ae-263148f0d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.065492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d417216-00df-42e1-91ae-263148f0d9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.092300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0d417216-00df-42e1-91ae-263148f0d9e3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:35.093568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0d417216-00df-42e1-91ae-263148f0d9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:35.101617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 969/1943] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:35.105917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d417216-00df-42e1-91ae-263148f0d9e3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.115981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d417216-00df-42e1-91ae-263148f0d9e3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.120800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.121834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.122120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.122968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.123622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:35.123926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.125686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.140769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 970/1944] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:54:35.145005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.156557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde8797-fde8-4333-a2a0-e226ed87f101 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.157310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:35.161602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e11fd6-cc23-4aeb-ad12-2094c9d9d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.162378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e11fd6-cc23-4aeb-ad12-2094c9d9d533 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.162780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e11fd6-cc23-4aeb-ad12-2094c9d9d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.163063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e11fd6-cc23-4aeb-ad12-2094c9d9d533 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.163694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e11fd6-cc23-4aeb-ad12-2094c9d9d533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.165671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e11fd6-cc23-4aeb-ad12-2094c9d9d533 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.294672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 971/1945] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:35 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:35.298543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e11fd6-cc23-4aeb-ad12-2094c9d9d533 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.301845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0539bdb7fa4635b36f22e3a5fc13ec'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.383424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0539bdb7fa4635b36f22e3a5fc13ec'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.384378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0539bdb7fa4635b36f22e3a5fc13ec'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.386470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e11fd6-cc23-4aeb-ad12-2094c9d9d533 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.398832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816f20d-b6ae-4cc8-848d-b930db7f79e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.398832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816f20d-b6ae-4cc8-848d-b930db7f79e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.398832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816f20d-b6ae-4cc8-848d-b930db7f79e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.398832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816f20d-b6ae-4cc8-848d-b930db7f79e3 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.399206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816f20d-b6ae-4cc8-848d-b930db7f79e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.399408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816f20d-b6ae-4cc8-848d-b930db7f79e3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.403515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0539bdb7fa4635b36f22e3a5fc13ec'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.411932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0539bdb7fa4635b36f22e3a5fc13ec'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.420230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 975/1946] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:34 2022] DELETE /identity/v3/users/7f0539bdb7fa4635b36f22e3a5fc13ec => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:35.495945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 972/1947] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:35 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:35.498332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.500427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:35.583468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf9769-68be-4455-9131-d0452e8a612a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.588660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3324c4-3993-4fd0-9087-153b55d52d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.589314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3324c4-3993-4fd0-9087-153b55d52d02 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.589560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3324c4-3993-4fd0-9087-153b55d52d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.589839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3324c4-3993-4fd0-9087-153b55d52d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.590581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3324c4-3993-4fd0-9087-153b55d52d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.592770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3324c4-3993-4fd0-9087-153b55d52d02 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.603539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 976/1948] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:35.608366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816f20d-b6ae-4cc8-848d-b930db7f79e3 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.618438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816f20d-b6ae-4cc8-848d-b930db7f79e3 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.623626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945928f-ba1f-45f2-a9df-b1037a3806ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.624316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945928f-ba1f-45f2-a9df-b1037a3806ae admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.624671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945928f-ba1f-45f2-a9df-b1037a3806ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.624957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945928f-ba1f-45f2-a9df-b1037a3806ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.625609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945928f-ba1f-45f2-a9df-b1037a3806ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.627457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945928f-ba1f-45f2-a9df-b1037a3806ae admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.650189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 973/1949] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:35.652358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3324c4-3993-4fd0-9087-153b55d52d02 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.663500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3324c4-3993-4fd0-9087-153b55d52d02 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.671682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.672878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.672878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.673053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] PATH_INFO: `/v3/projects/3a4ee86d812347c1881b3ef9e6778a38/users/f930a7593f0a48efa8ad7e0c8d5ecc2a/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.704052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4ee86d812347c1881b3ef9e6778a38, user_id=f930a7593f0a48efa8ad7e0c8d5ecc2a, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.707440 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.816431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 977/1950] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:35 2022] PUT /identity/v3/projects/3a4ee86d812347c1881b3ef9e6778a38/users/f930a7593f0a48efa8ad7e0c8d5ecc2a/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:35.824339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf4cda-4880-4a5e-9fd2-cd297f0130c7 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.824846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf4cda-4880-4a5e-9fd2-cd297f0130c7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.824990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf4cda-4880-4a5e-9fd2-cd297f0130c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.854007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3acf4cda-4880-4a5e-9fd2-cd297f0130c7 None None] MFA Rules not processed for user `f930a7593f0a48efa8ad7e0c8d5ecc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:35.855282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3acf4cda-4880-4a5e-9fd2-cd297f0130c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:35.870590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:35.899528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.903084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:35.956515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 974/1951] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:35.964302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf4cda-4880-4a5e-9fd2-cd297f0130c7 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.021135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f440cad-f31a-416d-b9c9-ad8d1cb8f193 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.028234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.029688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.030162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.030614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] PATH_INFO: `/v3/projects/8e63a1a71e0d412b96da76f1eb476fee` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.032535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e63a1a71e0d412b96da76f1eb476fee)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.035178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.296824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf4cda-4880-4a5e-9fd2-cd297f0130c7 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.296824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9b883-6216-4be1-aeee-51423856b8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.296824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9b883-6216-4be1-aeee-51423856b8ee admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.296824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9b883-6216-4be1-aeee-51423856b8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.299371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9b883-6216-4be1-aeee-51423856b8ee admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.299371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9b883-6216-4be1-aeee-51423856b8ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.299371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9b883-6216-4be1-aeee-51423856b8ee admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.496883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 975/1952] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:35 2022] POST /identity/v3/projects => generated 375 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:36.497710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e63a1a71e0d412b96da76f1eb476fee', 'request_id': 'req-284342f0-4adc-45a0-9af4-3d005e31db26'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.497710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e63a1a71e0d412b96da76f1eb476fee', 'request_id': 'req-284342f0-4adc-45a0-9af4-3d005e31db26'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.497897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9b883-6216-4be1-aeee-51423856b8ee admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.571991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9b883-6216-4be1-aeee-51423856b8ee admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.645918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:36.645918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e63a1a71e0d412b96da76f1eb476fee', 'request_id': 'req-284342f0-4adc-45a0-9af4-3d005e31db26'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.646285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.646919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.647205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.647480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.648124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.650284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.657154 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] The token cache is being invalidate because project 8e63a1a71e0d412b96da76f1eb476fee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:36.657627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.658276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-284342f0-4adc-45a0-9af4-3d005e31db26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.660171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 978/1953] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:35 2022] DELETE /identity/v3/projects/8e63a1a71e0d412b96da76f1eb476fee => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:36.665064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458b824-2d9d-4d84-9bc3-bcfcfb8a8b6e None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.665351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458b824-2d9d-4d84-9bc3-bcfcfb8a8b6e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.665631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458b824-2d9d-4d84-9bc3-bcfcfb8a8b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.687539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f458b824-2d9d-4d84-9bc3-bcfcfb8a8b6e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:36.688988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f458b824-2d9d-4d84-9bc3-bcfcfb8a8b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:36.697200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 979/1954] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:36.714007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548d04b-ea2e-4f82-8126-dd8b491b865f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.714007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548d04b-ea2e-4f82-8126-dd8b491b865f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.714007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548d04b-ea2e-4f82-8126-dd8b491b865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.726963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d548d04b-ea2e-4f82-8126-dd8b491b865f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:36.728153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d548d04b-ea2e-4f82-8126-dd8b491b865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:36.736005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 980/1955] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:36.741908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548d04b-ea2e-4f82-8126-dd8b491b865f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.752332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548d04b-ea2e-4f82-8126-dd8b491b865f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.755898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 976/1956] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:36 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:36.758781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.759408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.759518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.759747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.760107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.760733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:36.760995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.761604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:36.762756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.774912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b6381-802d-4f92-b7f8-499d53d16bea admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.777329 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 981/1957] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:54:36.782903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2efe4-f155-4930-a9d3-97082afe6371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.783055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2efe4-f155-4930-a9d3-97082afe6371 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.783351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2efe4-f155-4930-a9d3-97082afe6371 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.783598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2efe4-f155-4930-a9d3-97082afe6371 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.784258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2efe4-f155-4930-a9d3-97082afe6371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.785915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2efe4-f155-4930-a9d3-97082afe6371 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.786320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.800208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 977/1958] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:54:36.800877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704fc4a-e44f-4071-b920-520b6ac599d3 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.801703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2efe4-f155-4930-a9d3-97082afe6371 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.804112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257488a-ed00-448d-9c12-e54e6a63b144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.804891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257488a-ed00-448d-9c12-e54e6a63b144 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.805180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257488a-ed00-448d-9c12-e54e6a63b144 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.805457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257488a-ed00-448d-9c12-e54e6a63b144 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.806125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257488a-ed00-448d-9c12-e54e6a63b144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.808062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257488a-ed00-448d-9c12-e54e6a63b144 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.814277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2efe4-f155-4930-a9d3-97082afe6371 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.824431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f3694-f4bc-448a-86d2-5d51f27133f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.825208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3694-f4bc-448a-86d2-5d51f27133f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.826070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3694-f4bc-448a-86d2-5d51f27133f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.826381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3694-f4bc-448a-86d2-5d51f27133f9 admin admin] PATH_INFO: `/v3/projects/669d89eca0654489b76d550a858a97a4/users/03781dd96cbc4d2c9faabb5b03c9563c/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.847876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f3694-f4bc-448a-86d2-5d51f27133f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=669d89eca0654489b76d550a858a97a4, user_id=03781dd96cbc4d2c9faabb5b03c9563c, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.847876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f3694-f4bc-448a-86d2-5d51f27133f9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.960832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 982/1959] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:36 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:36.964250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257488a-ed00-448d-9c12-e54e6a63b144 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.970361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 978/1960] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:36 2022] PUT /identity/v3/projects/669d89eca0654489b76d550a858a97a4/users/03781dd96cbc4d2c9faabb5b03c9563c/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:36.974478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f3694-f4bc-448a-86d2-5d51f27133f9 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.038671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257488a-ed00-448d-9c12-e54e6a63b144 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.043876 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a33f-356a-4abd-bf2d-abead12f3de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.045069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a33f-356a-4abd-bf2d-abead12f3de8 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.045555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f3694-f4bc-448a-86d2-5d51f27133f9 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.046017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a33f-356a-4abd-bf2d-abead12f3de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.046349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a33f-356a-4abd-bf2d-abead12f3de8 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.052510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8a33f-356a-4abd-bf2d-abead12f3de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.052510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8a33f-356a-4abd-bf2d-abead12f3de8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.052704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa20f6d-94ff-4cad-8d41-af692951c3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.052704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa20f6d-94ff-4cad-8d41-af692951c3ae admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.052704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa20f6d-94ff-4cad-8d41-af692951c3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.052704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa20f6d-94ff-4cad-8d41-af692951c3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.053025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa20f6d-94ff-4cad-8d41-af692951c3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.054407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa20f6d-94ff-4cad-8d41-af692951c3ae admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.069553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 979/1961] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:54:37.070735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa20f6d-94ff-4cad-8d41-af692951c3ae admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.081118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa20f6d-94ff-4cad-8d41-af692951c3ae admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.085819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc74808-4525-4b25-a24c-ec428fb177b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.086510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc74808-4525-4b25-a24c-ec428fb177b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.086791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc74808-4525-4b25-a24c-ec428fb177b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.087072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc74808-4525-4b25-a24c-ec428fb177b6 admin admin] PATH_INFO: `/v3/projects/669d89eca0654489b76d550a858a97a4/users/03781dd96cbc4d2c9faabb5b03c9563c/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.089130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc74808-4525-4b25-a24c-ec428fb177b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=669d89eca0654489b76d550a858a97a4, user_id=03781dd96cbc4d2c9faabb5b03c9563c, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.091935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc74808-4525-4b25-a24c-ec428fb177b6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.102408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 980/1962] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:37 2022] PUT /identity/v3/projects/669d89eca0654489b76d550a858a97a4/users/03781dd96cbc4d2c9faabb5b03c9563c/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:37.112754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a70d4-5542-4ce7-9648-fb3a5baec36e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.113046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a70d4-5542-4ce7-9648-fb3a5baec36e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.113325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a70d4-5542-4ce7-9648-fb3a5baec36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.141963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:37.144810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-652a70d4-5542-4ce7-9648-fb3a5baec36e None None] MFA Rules not processed for user `03781dd96cbc4d2c9faabb5b03c9563c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:37.146172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-652a70d4-5542-4ce7-9648-fb3a5baec36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:37.173367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 983/1963] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:36 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:37.175297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a33f-356a-4abd-bf2d-abead12f3de8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.198503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a33f-356a-4abd-bf2d-abead12f3de8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.301386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 981/1964] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:37.307832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97f81c-4486-4f8b-aa09-aa422f2b6694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.308616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97f81c-4486-4f8b-aa09-aa422f2b6694 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.309294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97f81c-4486-4f8b-aa09-aa422f2b6694 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.309584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97f81c-4486-4f8b-aa09-aa422f2b6694 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.312815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97f81c-4486-4f8b-aa09-aa422f2b6694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.314747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97f81c-4486-4f8b-aa09-aa422f2b6694 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.315685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a70d4-5542-4ce7-9648-fb3a5baec36e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.317598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-652a70d4-5542-4ce7-9648-fb3a5baec36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:37.333904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 984/1965] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:37.334822 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97f81c-4486-4f8b-aa09-aa422f2b6694 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.352031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97f81c-4486-4f8b-aa09-aa422f2b6694 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.372383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e8c88-04dd-4dfc-a011-99d78bf19116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.372383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e8c88-04dd-4dfc-a011-99d78bf19116 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.372383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e8c88-04dd-4dfc-a011-99d78bf19116 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.372383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e8c88-04dd-4dfc-a011-99d78bf19116 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.373563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e8c88-04dd-4dfc-a011-99d78bf19116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.373563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e8c88-04dd-4dfc-a011-99d78bf19116 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.403111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 985/1966] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:37.411987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e8c88-04dd-4dfc-a011-99d78bf19116 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.413501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a70d4-5542-4ce7-9648-fb3a5baec36e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.419957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bce09a-d51b-4a28-8cc9-9648106ba6fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.419957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bce09a-d51b-4a28-8cc9-9648106ba6fa service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.419957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bce09a-d51b-4a28-8cc9-9648106ba6fa service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.419957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bce09a-d51b-4a28-8cc9-9648106ba6fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.421920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b3bce09a-d51b-4a28-8cc9-9648106ba6fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:37.425806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:37.429799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e8c88-04dd-4dfc-a011-99d78bf19116 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.434687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073e8d2-7c20-4f24-bbcc-7a1d8d3b7175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.436040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073e8d2-7c20-4f24-bbcc-7a1d8d3b7175 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.436396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073e8d2-7c20-4f24-bbcc-7a1d8d3b7175 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.436743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073e8d2-7c20-4f24-bbcc-7a1d8d3b7175 admin admin] PATH_INFO: `/v3/projects/12b6a2d244f24e308a2dab71502ed33f/users/4db7ca6ae0ca474f86cbc0187908cf18/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.453047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073e8d2-7c20-4f24-bbcc-7a1d8d3b7175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b6a2d244f24e308a2dab71502ed33f, user_id=4db7ca6ae0ca474f86cbc0187908cf18, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.455902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073e8d2-7c20-4f24-bbcc-7a1d8d3b7175 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.488303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bce09a-d51b-4a28-8cc9-9648106ba6fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.489899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bce09a-d51b-4a28-8cc9-9648106ba6fa service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.501812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 982/1967] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:37.534861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 986/1968] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:37 2022] PUT /identity/v3/projects/12b6a2d244f24e308a2dab71502ed33f/users/4db7ca6ae0ca474f86cbc0187908cf18/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:37.546324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d095-41b4-41a3-a31d-b96c438df743 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.546623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d095-41b4-41a3-a31d-b96c438df743 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.546906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d095-41b4-41a3-a31d-b96c438df743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.582465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-21c0d095-41b4-41a3-a31d-b96c438df743 None None] MFA Rules not processed for user `4db7ca6ae0ca474f86cbc0187908cf18`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:37.583759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-21c0d095-41b4-41a3-a31d-b96c438df743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:37.662850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 983/1969] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:37.667584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073e8d2-7c20-4f24-bbcc-7a1d8d3b7175 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.746542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073e8d2-7c20-4f24-bbcc-7a1d8d3b7175 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.752749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dddc5-ee50-4891-b963-71789e41eeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.754025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dddc5-ee50-4891-b963-71789e41eeee admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.754025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dddc5-ee50-4891-b963-71789e41eeee admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.754025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dddc5-ee50-4891-b963-71789e41eeee admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.757912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dddc5-ee50-4891-b963-71789e41eeee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.757912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dddc5-ee50-4891-b963-71789e41eeee admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.781449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:37.878180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:37.886373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 987/1970] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:37 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:37.891072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0d095-41b4-41a3-a31d-b96c438df743 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.911785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0d095-41b4-41a3-a31d-b96c438df743 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.969692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8cc9-5f53-4257-8b93-c3d6ba28ca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.970347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8cc9-5f53-4257-8b93-c3d6ba28ca63 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.970640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8cc9-5f53-4257-8b93-c3d6ba28ca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.970903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8cc9-5f53-4257-8b93-c3d6ba28ca63 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.971544 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a8cc9-5f53-4257-8b93-c3d6ba28ca63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.973623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a8cc9-5f53-4257-8b93-c3d6ba28ca63 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:38.107889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 984/1971] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:37 2022] POST /identity/v3/users => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:38.108001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dddc5-ee50-4891-b963-71789e41eeee admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:38.114847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dddc5-ee50-4891-b963-71789e41eeee admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:38.119567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f57d-6275-46dc-af84-c87fc9dd2232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:38.120182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f57d-6275-46dc-af84-c87fc9dd2232 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:38.120467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f57d-6275-46dc-af84-c87fc9dd2232 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:38.120746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14f57d-6275-46dc-af84-c87fc9dd2232 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:38.121523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14f57d-6275-46dc-af84-c87fc9dd2232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:38.123214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14f57d-6275-46dc-af84-c87fc9dd2232 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:38.135861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 988/1972] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:38.141002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8cc9-5f53-4257-8b93-c3d6ba28ca63 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:38.151161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8cc9-5f53-4257-8b93-c3d6ba28ca63 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:38.156777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9458-56f1-420b-841a-af2cd6b76f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:38.157443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9458-56f1-420b-841a-af2cd6b76f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:38.157723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9458-56f1-420b-841a-af2cd6b76f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:38.157995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9458-56f1-420b-841a-af2cd6b76f26 admin admin] PATH_INFO: `/v3/projects/b30cf562d973440292978310ff1711da/users/1b26af5c87b74a09b6fe8132e6b90452/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:38.173696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d9458-56f1-420b-841a-af2cd6b76f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30cf562d973440292978310ff1711da, user_id=1b26af5c87b74a09b6fe8132e6b90452, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:38.176663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d9458-56f1-420b-841a-af2cd6b76f26 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:38.270871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 985/1973] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:38 2022] PUT /identity/v3/projects/b30cf562d973440292978310ff1711da/users/1b26af5c87b74a09b6fe8132e6b90452/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:38.273460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f57d-6275-46dc-af84-c87fc9dd2232 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:38.285768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14f57d-6275-46dc-af84-c87fc9dd2232 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:38.384815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f90a7c-0127-444f-9885-f209d7de020a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:38.385686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a7c-0127-444f-9885-f209d7de020a admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:38.386270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a7c-0127-444f-9885-f209d7de020a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:38.386707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a7c-0127-444f-9885-f209d7de020a admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:38.387546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f90a7c-0127-444f-9885-f209d7de020a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:38.390153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f90a7c-0127-444f-9885-f209d7de020a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:38.428067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 989/1974] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:38.430332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9458-56f1-420b-841a-af2cd6b76f26 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:38.431240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:38.452458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9458-56f1-420b-841a-af2cd6b76f26 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:38.463825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:38.464978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:38.465492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:38.466062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] PATH_INFO: `/v3/projects/b30cf562d973440292978310ff1711da/users/1b26af5c87b74a09b6fe8132e6b90452/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:38.471810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30cf562d973440292978310ff1711da, user_id=1b26af5c87b74a09b6fe8132e6b90452, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:38.476088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:38.491216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 986/1975] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:38 2022] PUT /identity/v3/projects/b30cf562d973440292978310ff1711da/users/1b26af5c87b74a09b6fe8132e6b90452/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:38.509673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bef499-af9f-45dc-80b3-aff91debadc1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:38.509673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bef499-af9f-45dc-80b3-aff91debadc1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:38.510013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bef499-af9f-45dc-80b3-aff91debadc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:38.550123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:38.553716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-46bef499-af9f-45dc-80b3-aff91debadc1 None None] MFA Rules not processed for user `1b26af5c87b74a09b6fe8132e6b90452`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:38.555334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-46bef499-af9f-45dc-80b3-aff91debadc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:38.695910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 990/1976] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:38.722557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:38.724442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:38.789735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c0a4-a0ed-4d5b-8073-c57ab3bbf7d6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:38.796104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efffe65-44ca-41d4-a97a-81bc82ed6a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:38.796422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efffe65-44ca-41d4-a97a-81bc82ed6a8d service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:38.796665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efffe65-44ca-41d4-a97a-81bc82ed6a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:38.796963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efffe65-44ca-41d4-a97a-81bc82ed6a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:38.882565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efffe65-44ca-41d4-a97a-81bc82ed6a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:38.884107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efffe65-44ca-41d4-a97a-81bc82ed6a8d service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:38.899708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 987/1977] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:39.094202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:39.692107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:39.831462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.124064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.144690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.168797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.241166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.379808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.540088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.580514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.745436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.774756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.904191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.962774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:41.208294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:41.282438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:41.328025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:41.373816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:41.396482 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:47.299373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bef499-af9f-45dc-80b3-aff91debadc1 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:47.310316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:47.400840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bef499-af9f-45dc-80b3-aff91debadc1 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:47.405574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:47.406254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:47.406526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:47.406893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] PATH_INFO: `/v3/users/f930a7593f0a48efa8ad7e0c8d5ecc2a` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:47.408043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f930a7593f0a48efa8ad7e0c8d5ecc2a)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:47.410065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:47.587528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f930a7593f0a48efa8ad7e0c8d5ecc2a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:47.740517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f930a7593f0a48efa8ad7e0c8d5ecc2a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:47.741653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f930a7593f0a48efa8ad7e0c8d5ecc2a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:47.756789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f930a7593f0a48efa8ad7e0c8d5ecc2a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:47.766404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f930a7593f0a48efa8ad7e0c8d5ecc2a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:47.773086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 991/1978] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:47 2022] DELETE /identity/v3/users/f930a7593f0a48efa8ad7e0c8d5ecc2a => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:47.777730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e6c62-8d70-415b-adf6-d797535be1b7 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:47.778291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e6c62-8d70-415b-adf6-d797535be1b7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:47.778436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e6c62-8d70-415b-adf6-d797535be1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:47.799258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-077e6c62-8d70-415b-adf6-d797535be1b7 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:47.800466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-077e6c62-8d70-415b-adf6-d797535be1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:47.860691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 988/1979] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:47.870424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:47.872491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:47.938875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0234b4-f559-4612-a7cc-9fa8d3a18ea4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:47.942527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da725daf-e461-473f-a8cb-77d9c52cac1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:47.943139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da725daf-e461-473f-a8cb-77d9c52cac1b service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:47.943419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da725daf-e461-473f-a8cb-77d9c52cac1b service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:47.943695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da725daf-e461-473f-a8cb-77d9c52cac1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:47.955237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da725daf-e461-473f-a8cb-77d9c52cac1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:47.957376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da725daf-e461-473f-a8cb-77d9c52cac1b service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:47.967418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 992/1980] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:48.414019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e6c62-8d70-415b-adf6-d797535be1b7 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:48.415880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-077e6c62-8d70-415b-adf6-d797535be1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:48.434386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e6c62-8d70-415b-adf6-d797535be1b7 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:48.441671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:48.442715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:48.443261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:48.443706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] PATH_INFO: `/v3/projects/3a4ee86d812347c1881b3ef9e6778a38` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:48.444961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4ee86d812347c1881b3ef9e6778a38)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:48.447370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:48.581000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4ee86d812347c1881b3ef9e6778a38', 'request_id': 'req-821c474a-2342-4795-89f2-655dd576f08b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:48.587537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4ee86d812347c1881b3ef9e6778a38', 'request_id': 'req-821c474a-2342-4795-89f2-655dd576f08b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:48.677504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4ee86d812347c1881b3ef9e6778a38', 'request_id': 'req-821c474a-2342-4795-89f2-655dd576f08b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:48.689917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] The token cache is being invalidate because project 3a4ee86d812347c1881b3ef9e6778a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:48.690242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:48.691062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:48.692691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 989/1981] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:48 2022] DELETE /identity/v3/projects/3a4ee86d812347c1881b3ef9e6778a38 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:48.702161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da725daf-e461-473f-a8cb-77d9c52cac1b service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:48.712234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da725daf-e461-473f-a8cb-77d9c52cac1b service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:48.741887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:48.771276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:48.771910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:48.772221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:48.772492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] PATH_INFO: `/v3/users/03781dd96cbc4d2c9faabb5b03c9563c` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:48.773608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03781dd96cbc4d2c9faabb5b03c9563c)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:48.775558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:48.994275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03781dd96cbc4d2c9faabb5b03c9563c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.065431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03781dd96cbc4d2c9faabb5b03c9563c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.066173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03781dd96cbc4d2c9faabb5b03c9563c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.081829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03781dd96cbc4d2c9faabb5b03c9563c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.088820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03781dd96cbc4d2c9faabb5b03c9563c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.095484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 993/1982] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:48 2022] DELETE /identity/v3/users/03781dd96cbc4d2c9faabb5b03c9563c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:49.559696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:49.562683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:49.594842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c474a-2342-4795-89f2-655dd576f08b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:49.662274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:49.663057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:49.663464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:49.663839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] PATH_INFO: `/v3/projects/669d89eca0654489b76d550a858a97a4` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:49.665175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=669d89eca0654489b76d550a858a97a4)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:49.667314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:49.684248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:49.793696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669d89eca0654489b76d550a858a97a4', 'request_id': 'req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.800376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669d89eca0654489b76d550a858a97a4', 'request_id': 'req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.823298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:49.824679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:49.879787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669d89eca0654489b76d550a858a97a4', 'request_id': 'req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.891177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7026fa-3211-49e2-b2ce-10e3a5a2b578 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:49.895088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] The token cache is being invalidate because project 669d89eca0654489b76d550a858a97a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:49.895567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.896389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40d573ff-d7eb-4a4b-9d0f-fd04f832d412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:49.898931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 990/1983] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:49 2022] DELETE /identity/v3/projects/669d89eca0654489b76d550a858a97a4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:49.903552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc159db9-9105-4fc9-9827-5ae0484ad4c7 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:49.903842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc159db9-9105-4fc9-9827-5ae0484ad4c7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:49.904472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc159db9-9105-4fc9-9827-5ae0484ad4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:49.943971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-fc159db9-9105-4fc9-9827-5ae0484ad4c7 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:49.945331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fc159db9-9105-4fc9-9827-5ae0484ad4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:49.959986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:49.960596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:49.960879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:49.961154 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:49.963402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:49.987236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:50.015062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 991/1984] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:50.023718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e26f6d-8ab0-4b9c-8a7f-e848d02e6b7a None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.024003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e26f6d-8ab0-4b9c-8a7f-e848d02e6b7a None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.024276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e26f6d-8ab0-4b9c-8a7f-e848d02e6b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.040912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:50.042529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:50.046396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-81e26f6d-8ab0-4b9c-8a7f-e848d02e6b7a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:50.049407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-81e26f6d-8ab0-4b9c-8a7f-e848d02e6b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:50.053386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 994/1985] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:50.055907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 992/1986] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:50.060340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:50.071085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a38b1-7cae-4359-88fa-42cb0401f0e8 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:50.076258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:50.076877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.077156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.077458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.078113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:50.078454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:50.080051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:50.094324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 995/1987] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:50.104329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e26f6d-8ab0-4b9c-8a7f-e848d02e6b7a None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:50.110531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e26f6d-8ab0-4b9c-8a7f-e848d02e6b7a None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:50.116757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb53a02-5019-49dd-af92-8fd6b7ec3185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:50.117371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb53a02-5019-49dd-af92-8fd6b7ec3185 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.117650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb53a02-5019-49dd-af92-8fd6b7ec3185 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.117950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb53a02-5019-49dd-af92-8fd6b7ec3185 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.118714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb53a02-5019-49dd-af92-8fd6b7ec3185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:50.121152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb53a02-5019-49dd-af92-8fd6b7ec3185 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:50.215884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 993/1988] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:50 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:50.220592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:50.230452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5dd3-0ae1-4604-bead-5a7f4d085e34 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:50.287439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e47c8a-8874-414d-9509-3530aca3ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:50.288006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e47c8a-8874-414d-9509-3530aca3ff7a admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.288546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e47c8a-8874-414d-9509-3530aca3ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.288996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e47c8a-8874-414d-9509-3530aca3ff7a admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.290328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e47c8a-8874-414d-9509-3530aca3ff7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:50.292462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e47c8a-8874-414d-9509-3530aca3ff7a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:50.404988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 996/1989] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:50 2022] POST /identity/v3/users => generated 282 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:50.409615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb53a02-5019-49dd-af92-8fd6b7ec3185 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:50.422965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb53a02-5019-49dd-af92-8fd6b7ec3185 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:50.427599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d5c3-c89b-4954-80c8-0cec057b15a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:50.430078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d5c3-c89b-4954-80c8-0cec057b15a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.430078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d5c3-c89b-4954-80c8-0cec057b15a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.430078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d5c3-c89b-4954-80c8-0cec057b15a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.430078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39d5c3-c89b-4954-80c8-0cec057b15a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:50.431138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39d5c3-c89b-4954-80c8-0cec057b15a6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:50.441915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 994/1990] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:50.446101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e47c8a-8874-414d-9509-3530aca3ff7a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:50.462756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e47c8a-8874-414d-9509-3530aca3ff7a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:50.470963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce287543-224a-4a1f-ae5d-9478b8304937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:50.479044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce287543-224a-4a1f-ae5d-9478b8304937 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.479044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce287543-224a-4a1f-ae5d-9478b8304937 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.479044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce287543-224a-4a1f-ae5d-9478b8304937 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.479044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce287543-224a-4a1f-ae5d-9478b8304937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:50.479044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce287543-224a-4a1f-ae5d-9478b8304937 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:50.496093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 997/1991] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:50.500987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d5c3-c89b-4954-80c8-0cec057b15a6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:50.511835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d5c3-c89b-4954-80c8-0cec057b15a6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:50.519646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:50.520685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.522057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.522057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] PATH_INFO: `/v3/projects/1ba8fd6038c742da8fbd814b9d1f02ba/users/65ed045641eb44389410527d391a22db/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.549675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba8fd6038c742da8fbd814b9d1f02ba, user_id=65ed045641eb44389410527d391a22db, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:50.552836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:50.670951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 995/1992] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:50 2022] PUT /identity/v3/projects/1ba8fd6038c742da8fbd814b9d1f02ba/users/65ed045641eb44389410527d391a22db/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:50.682164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0343-f3fe-49ec-a9f5-05cd688e7d2d None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.682672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0343-f3fe-49ec-a9f5-05cd688e7d2d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.682955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0343-f3fe-49ec-a9f5-05cd688e7d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.714691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a8bd0343-f3fe-49ec-a9f5-05cd688e7d2d None None] MFA Rules not processed for user `65ed045641eb44389410527d391a22db`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:50.716294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a8bd0343-f3fe-49ec-a9f5-05cd688e7d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:50.812138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 998/1993] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:50.823899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:50.825713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:50.871542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:50.886903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7d001-220c-4f4a-ae4c-845fda616915 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:50.892124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8189d8-3a4f-4d8a-a433-21a67e87baac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:50.892868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8189d8-3a4f-4d8a-a433-21a67e87baac service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.893245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8189d8-3a4f-4d8a-a433-21a67e87baac service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.893737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8189d8-3a4f-4d8a-a433-21a67e87baac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.906090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8189d8-3a4f-4d8a-a433-21a67e87baac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:50.907744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8189d8-3a4f-4d8a-a433-21a67e87baac service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:50.917380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 996/1994] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:51.803043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd0343-f3fe-49ec-a9f5-05cd688e7d2d None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:51.806013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a8bd0343-f3fe-49ec-a9f5-05cd688e7d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:51.825965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:51.886853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd0343-f3fe-49ec-a9f5-05cd688e7d2d None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:51.892648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:51.893300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:51.893580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:51.893855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] PATH_INFO: `/v3/users/1f526f3b5020430a8a810a083256798b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:51.895023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f526f3b5020430a8a810a083256798b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:51.896993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:52.170996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f526f3b5020430a8a810a083256798b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:52.266654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f526f3b5020430a8a810a083256798b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:52.267277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f526f3b5020430a8a810a083256798b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:52.279922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f526f3b5020430a8a810a083256798b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:52.287495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f526f3b5020430a8a810a083256798b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:52.293663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 999/1995] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:51 2022] DELETE /identity/v3/users/1f526f3b5020430a8a810a083256798b => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:52.298778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45d2d8-485e-48c5-b11f-3adde8772425 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:52.299233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45d2d8-485e-48c5-b11f-3adde8772425 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:52.299352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45d2d8-485e-48c5-b11f-3adde8772425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:52.318261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ac45d2d8-485e-48c5-b11f-3adde8772425 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:52.319499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ac45d2d8-485e-48c5-b11f-3adde8772425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:52.378847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 997/1996] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:52.388776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:52.391010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:52.455177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70517595-1859-4a3c-ac0f-e548b6af4d17 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:52.458918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c1ca-178b-4574-9933-1a1f5f7123a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:52.459535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c1ca-178b-4574-9933-1a1f5f7123a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:52.459797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c1ca-178b-4574-9933-1a1f5f7123a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:52.460067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594c1ca-178b-4574-9933-1a1f5f7123a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:52.470374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594c1ca-178b-4574-9933-1a1f5f7123a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:52.472129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594c1ca-178b-4574-9933-1a1f5f7123a5 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:52.483663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1000/1997] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:53.020766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45d2d8-485e-48c5-b11f-3adde8772425 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:53.023720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ac45d2d8-485e-48c5-b11f-3adde8772425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:53.051422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45d2d8-485e-48c5-b11f-3adde8772425 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:53.057601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:53.058699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:53.059054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:53.059338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] PATH_INFO: `/v3/projects/0ed33b6b61ed4d7ca2167695c42d9ba4` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:53.060549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ed33b6b61ed4d7ca2167695c42d9ba4)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:53.062692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:53.194342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ed33b6b61ed4d7ca2167695c42d9ba4', 'request_id': 'req-8ddf50d1-0724-4948-8bc2-015290f8de25'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.201376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ed33b6b61ed4d7ca2167695c42d9ba4', 'request_id': 'req-8ddf50d1-0724-4948-8bc2-015290f8de25'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.283218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ed33b6b61ed4d7ca2167695c42d9ba4', 'request_id': 'req-8ddf50d1-0724-4948-8bc2-015290f8de25'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.299531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] The token cache is being invalidate because project 0ed33b6b61ed4d7ca2167695c42d9ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:53.299531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.299531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.299531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 998/1998] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:53 2022] DELETE /identity/v3/projects/0ed33b6b61ed4d7ca2167695c42d9ba4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:53.302283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c1ca-178b-4574-9933-1a1f5f7123a5 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:53.314166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594c1ca-178b-4574-9933-1a1f5f7123a5 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:53.348083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:53.386232 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:53.387100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:53.387346 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:53.387622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] PATH_INFO: `/v3/users/766c4bad8ed04813887811a2ea526cbf` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:53.388800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=766c4bad8ed04813887811a2ea526cbf)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:53.391244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:53.613592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '766c4bad8ed04813887811a2ea526cbf'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.746639 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '766c4bad8ed04813887811a2ea526cbf'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.748135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '766c4bad8ed04813887811a2ea526cbf'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.763625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '766c4bad8ed04813887811a2ea526cbf'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.772989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0465367a-4b02-4860-98bb-928e266304c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '766c4bad8ed04813887811a2ea526cbf'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:53.778250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1001/1999] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:53 2022] DELETE /identity/v3/users/766c4bad8ed04813887811a2ea526cbf => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:54.277316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:54.282389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:54.306978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf50d1-0724-4948-8bc2-015290f8de25 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:54.378463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:54.379112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:54.379397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:54.379673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] PATH_INFO: `/v3/projects/0aeb951ef3c04526b8a3d860edfed03a` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:54.380849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aeb951ef3c04526b8a3d860edfed03a)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:54.382809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:54.392999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:54.515592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aeb951ef3c04526b8a3d860edfed03a', 'request_id': 'req-7709c19a-c883-4ec8-8e7b-25d88f215f51'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:54.522209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aeb951ef3c04526b8a3d860edfed03a', 'request_id': 'req-7709c19a-c883-4ec8-8e7b-25d88f215f51'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:54.619779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aeb951ef3c04526b8a3d860edfed03a', 'request_id': 'req-7709c19a-c883-4ec8-8e7b-25d88f215f51'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:54.628152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] The token cache is being invalidate because project 0aeb951ef3c04526b8a3d860edfed03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:54.628520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:54.629272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7709c19a-c883-4ec8-8e7b-25d88f215f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:54.631322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 999/2000] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:54 2022] DELETE /identity/v3/projects/0aeb951ef3c04526b8a3d860edfed03a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:54.636084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19fdb-fa65-4185-985e-f1ea67997982 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:54.636603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19fdb-fa65-4185-985e-f1ea67997982 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:54.636703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19fdb-fa65-4185-985e-f1ea67997982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:54.660087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-37c19fdb-fa65-4185-985e-f1ea67997982 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:54.661343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-37c19fdb-fa65-4185-985e-f1ea67997982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:54.724563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1002/2001] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:54.732596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cb719-e4ab-47d7-94d6-2d83cdceab37 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:54.732887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cb719-e4ab-47d7-94d6-2d83cdceab37 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:54.733723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cb719-e4ab-47d7-94d6-2d83cdceab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:54.754326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-015cb719-e4ab-47d7-94d6-2d83cdceab37 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:54.759154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-015cb719-e4ab-47d7-94d6-2d83cdceab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:54.763791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1000/2002] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:54.768529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c19fdb-fa65-4185-985e-f1ea67997982 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:54.778688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c19fdb-fa65-4185-985e-f1ea67997982 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:54.782941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:54.787818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:54.787818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:54.787818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:54.787818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:54.787818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:54.787818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:54.799311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1003/2003] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:54.804004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cb719-e4ab-47d7-94d6-2d83cdceab37 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:54.814240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cb719-e4ab-47d7-94d6-2d83cdceab37 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:54.825411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ec45-45ef-4d2b-a0f3-51e828b21606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:54.825411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ec45-45ef-4d2b-a0f3-51e828b21606 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:54.825411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ec45-45ef-4d2b-a0f3-51e828b21606 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:54.825411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ec45-45ef-4d2b-a0f3-51e828b21606 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:54.825801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6ec45-45ef-4d2b-a0f3-51e828b21606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:54.825801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6ec45-45ef-4d2b-a0f3-51e828b21606 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:54.908133 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1001/2004] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:54 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:54.912626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:54.923061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9b54c-02d0-451b-b9fd-810a2f8a4ff9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:54.962192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:54.980835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8efdc-e4b2-4d16-9a1f-a126e358bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:54.981449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8efdc-e4b2-4d16-9a1f-a126e358bfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:54.981733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8efdc-e4b2-4d16-9a1f-a126e358bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:54.982011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8efdc-e4b2-4d16-9a1f-a126e358bfa5 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:54.982811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8efdc-e4b2-4d16-9a1f-a126e358bfa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:54.984784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8efdc-e4b2-4d16-9a1f-a126e358bfa5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:55.100666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1004/2005] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:54 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:55.104849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ec45-45ef-4d2b-a0f3-51e828b21606 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:55.115276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ec45-45ef-4d2b-a0f3-51e828b21606 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:55.119996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c36a58-8d18-467f-90dd-188ca643cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:55.120602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36a58-8d18-467f-90dd-188ca643cc51 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.120925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36a58-8d18-467f-90dd-188ca643cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.121198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36a58-8d18-467f-90dd-188ca643cc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.125685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c36a58-8d18-467f-90dd-188ca643cc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:55.127637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c36a58-8d18-467f-90dd-188ca643cc51 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:55.140304 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1002/2006] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:55.144774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8efdc-e4b2-4d16-9a1f-a126e358bfa5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:55.156427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8efdc-e4b2-4d16-9a1f-a126e358bfa5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:55.162323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178d20d-a4e3-4706-802e-50b8f39f3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:55.162952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178d20d-a4e3-4706-802e-50b8f39f3ecd admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.163332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178d20d-a4e3-4706-802e-50b8f39f3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.163664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178d20d-a4e3-4706-802e-50b8f39f3ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.164389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178d20d-a4e3-4706-802e-50b8f39f3ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:55.166434 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178d20d-a4e3-4706-802e-50b8f39f3ecd admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:55.185808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1005/2007] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:54:55.187921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c36a58-8d18-467f-90dd-188ca643cc51 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:55.198693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c36a58-8d18-467f-90dd-188ca643cc51 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:55.204413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27daf48-6e01-4274-a0b3-5882e9c39e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:55.205173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27daf48-6e01-4274-a0b3-5882e9c39e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.205516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27daf48-6e01-4274-a0b3-5882e9c39e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.205856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27daf48-6e01-4274-a0b3-5882e9c39e97 admin admin] PATH_INFO: `/v3/projects/e220224633104bcb865ddfab9e7fe033/users/62a404305ab046bea4d87a4d75bf203c/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.221386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27daf48-6e01-4274-a0b3-5882e9c39e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e220224633104bcb865ddfab9e7fe033, user_id=62a404305ab046bea4d87a4d75bf203c, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:55.224085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27daf48-6e01-4274-a0b3-5882e9c39e97 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:55.379066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1003/2008] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:55 2022] PUT /identity/v3/projects/e220224633104bcb865ddfab9e7fe033/users/62a404305ab046bea4d87a4d75bf203c/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:55.390504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64add2a-8af7-4c1a-bf6d-b765a23f783b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.391148 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64add2a-8af7-4c1a-bf6d-b765a23f783b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.391488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64add2a-8af7-4c1a-bf6d-b765a23f783b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.424880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c64add2a-8af7-4c1a-bf6d-b765a23f783b None None] MFA Rules not processed for user `62a404305ab046bea4d87a4d75bf203c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:55.426324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c64add2a-8af7-4c1a-bf6d-b765a23f783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:55.521386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1006/2009] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:55.521509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27daf48-6e01-4274-a0b3-5882e9c39e97 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:55.578732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27daf48-6e01-4274-a0b3-5882e9c39e97 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:55.590579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb0cb9-3383-4346-8dd0-8c9addd8de6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:55.590579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb0cb9-3383-4346-8dd0-8c9addd8de6a admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.590579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb0cb9-3383-4346-8dd0-8c9addd8de6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.590579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb0cb9-3383-4346-8dd0-8c9addd8de6a admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.591142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb0cb9-3383-4346-8dd0-8c9addd8de6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:55.591142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb0cb9-3383-4346-8dd0-8c9addd8de6a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:55.722576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1004/2010] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:55 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:55.723713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64add2a-8af7-4c1a-bf6d-b765a23f783b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:55.734298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64add2a-8af7-4c1a-bf6d-b765a23f783b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:55.793305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8368444-faf3-4055-9c34-8606d9ee2688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:55.793828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8368444-faf3-4055-9c34-8606d9ee2688 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.794103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8368444-faf3-4055-9c34-8606d9ee2688 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.794568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8368444-faf3-4055-9c34-8606d9ee2688 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.795210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8368444-faf3-4055-9c34-8606d9ee2688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:55.797873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8368444-faf3-4055-9c34-8606d9ee2688 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:55.917621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1007/2011] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:55 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:55.922922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb0cb9-3383-4346-8dd0-8c9addd8de6a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:55.933488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb0cb9-3383-4346-8dd0-8c9addd8de6a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:55.939625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05136dd-86ea-49fb-a1bc-2b971ccf125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:55.939625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05136dd-86ea-49fb-a1bc-2b971ccf125e admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.939841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05136dd-86ea-49fb-a1bc-2b971ccf125e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.940035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05136dd-86ea-49fb-a1bc-2b971ccf125e admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.940691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05136dd-86ea-49fb-a1bc-2b971ccf125e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:55.942344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05136dd-86ea-49fb-a1bc-2b971ccf125e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:55.954200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1005/2012] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:55.958863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8368444-faf3-4055-9c34-8606d9ee2688 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:55.971438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8368444-faf3-4055-9c34-8606d9ee2688 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:55.977485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb4e5-f707-4dd6-adcc-15a06d1e10b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:55.978293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb4e5-f707-4dd6-adcc-15a06d1e10b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.978586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb4e5-f707-4dd6-adcc-15a06d1e10b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.978864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb4e5-f707-4dd6-adcc-15a06d1e10b5 admin admin] PATH_INFO: `/v3/projects/05f822df365b4acbb0ba69cc4696bf80/users/514e8cc06e094d6b9e15eaa010194caa/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.995400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121eb4e5-f707-4dd6-adcc-15a06d1e10b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f822df365b4acbb0ba69cc4696bf80, user_id=514e8cc06e094d6b9e15eaa010194caa, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:55.998107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121eb4e5-f707-4dd6-adcc-15a06d1e10b5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:56.073366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1008/2013] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:55 2022] PUT /identity/v3/projects/05f822df365b4acbb0ba69cc4696bf80/users/514e8cc06e094d6b9e15eaa010194caa/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:56.077853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05136dd-86ea-49fb-a1bc-2b971ccf125e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:56.088405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05136dd-86ea-49fb-a1bc-2b971ccf125e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:56.092360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:56.175153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad62cb-e298-4381-a361-c65e16ceb99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:56.175780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad62cb-e298-4381-a361-c65e16ceb99b admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:56.176061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad62cb-e298-4381-a361-c65e16ceb99b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:56.176334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad62cb-e298-4381-a361-c65e16ceb99b admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:56.177168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad62cb-e298-4381-a361-c65e16ceb99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:56.178892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad62cb-e298-4381-a361-c65e16ceb99b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:56.189115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1006/2014] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:56.193330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb4e5-f707-4dd6-adcc-15a06d1e10b5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:56.205034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb4e5-f707-4dd6-adcc-15a06d1e10b5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:56.210250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:56.210915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:56.211246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:56.211524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] PATH_INFO: `/v3/projects/05f822df365b4acbb0ba69cc4696bf80/users/514e8cc06e094d6b9e15eaa010194caa/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:56.213602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f822df365b4acbb0ba69cc4696bf80, user_id=514e8cc06e094d6b9e15eaa010194caa, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:56.216348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:56.223735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:56.228981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1009/2015] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:56 2022] PUT /identity/v3/projects/05f822df365b4acbb0ba69cc4696bf80/users/514e8cc06e094d6b9e15eaa010194caa/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:56.241739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338f86-0f8a-4f1d-a612-896651d3fd82 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:56.242013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338f86-0f8a-4f1d-a612-896651d3fd82 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:56.242325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338f86-0f8a-4f1d-a612-896651d3fd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:56.275389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-46338f86-0f8a-4f1d-a612-896651d3fd82 None None] MFA Rules not processed for user `514e8cc06e094d6b9e15eaa010194caa`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:56.276639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-46338f86-0f8a-4f1d-a612-896651d3fd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:56.368947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1007/2016] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:56.381140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:56.383061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:56.458195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3308f-8d68-4dbb-872e-1e3c3dfa85e4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:56.462022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf606b-c8dd-4246-ad4b-5ce2f6a46b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:56.462690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf606b-c8dd-4246-ad4b-5ce2f6a46b21 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:56.462973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf606b-c8dd-4246-ad4b-5ce2f6a46b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:56.463249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf606b-c8dd-4246-ad4b-5ce2f6a46b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:56.537079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf606b-c8dd-4246-ad4b-5ce2f6a46b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:56.538569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf606b-c8dd-4246-ad4b-5ce2f6a46b21 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:56.550748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1010/2017] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:57.398410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46338f86-0f8a-4f1d-a612-896651d3fd82 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:57.399861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-46338f86-0f8a-4f1d-a612-896651d3fd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:57.488776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46338f86-0f8a-4f1d-a612-896651d3fd82 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:57.493586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:57.494263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:57.494569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:57.494830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] PATH_INFO: `/v3/users/4db7ca6ae0ca474f86cbc0187908cf18` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:57.495992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db7ca6ae0ca474f86cbc0187908cf18)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:57.498049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:57.709638 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:57.719993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db7ca6ae0ca474f86cbc0187908cf18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:57.816676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db7ca6ae0ca474f86cbc0187908cf18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:57.816676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db7ca6ae0ca474f86cbc0187908cf18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:57.816676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db7ca6ae0ca474f86cbc0187908cf18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:57.822715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db7ca6ae0ca474f86cbc0187908cf18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:57.840471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1008/2018] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:57 2022] DELETE /identity/v3/users/4db7ca6ae0ca474f86cbc0187908cf18 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:57.848684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c6206-1cc7-410c-bc9a-0a7b598f476f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:57.849151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c6206-1cc7-410c-bc9a-0a7b598f476f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:57.849262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c6206-1cc7-410c-bc9a-0a7b598f476f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:57.868582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-501c6206-1cc7-410c-bc9a-0a7b598f476f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:57.869780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-501c6206-1cc7-410c-bc9a-0a7b598f476f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:57.932239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1011/2019] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:57.939361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:57.940774 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:58.001183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb5bbf-2828-43a5-86fb-6eb8b6660cee admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:58.004926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582cf67-797e-4de9-aae9-25c09b8b88a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:58.005877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582cf67-797e-4de9-aae9-25c09b8b88a6 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:58.005877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582cf67-797e-4de9-aae9-25c09b8b88a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:58.006111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582cf67-797e-4de9-aae9-25c09b8b88a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:58.015710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582cf67-797e-4de9-aae9-25c09b8b88a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:58.017529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582cf67-797e-4de9-aae9-25c09b8b88a6 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:58.027462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1009/2020] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:58.473525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c6206-1cc7-410c-bc9a-0a7b598f476f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:58.475077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-501c6206-1cc7-410c-bc9a-0a7b598f476f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:58.489731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c6206-1cc7-410c-bc9a-0a7b598f476f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:58.494735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:58.495304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:58.496092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:58.496368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] PATH_INFO: `/v3/projects/12b6a2d244f24e308a2dab71502ed33f` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:58.497692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b6a2d244f24e308a2dab71502ed33f)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:58.499734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:58.612222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b6a2d244f24e308a2dab71502ed33f', 'request_id': 'req-358254e6-0dcf-4643-b848-27dd70b34b0c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:58.613370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:58.620063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b6a2d244f24e308a2dab71502ed33f', 'request_id': 'req-358254e6-0dcf-4643-b848-27dd70b34b0c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:58.683422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b6a2d244f24e308a2dab71502ed33f', 'request_id': 'req-358254e6-0dcf-4643-b848-27dd70b34b0c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:58.696187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] The token cache is being invalidate because project 12b6a2d244f24e308a2dab71502ed33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:58.699810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:58.699810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:58.699810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1012/2021] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:58 2022] DELETE /identity/v3/projects/12b6a2d244f24e308a2dab71502ed33f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:58.702960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582cf67-797e-4de9-aae9-25c09b8b88a6 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:58.714971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582cf67-797e-4de9-aae9-25c09b8b88a6 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:58.792850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:58.793497 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:58.793781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:58.794063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] PATH_INFO: `/v3/users/1b26af5c87b74a09b6fe8132e6b90452` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:58.795288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b26af5c87b74a09b6fe8132e6b90452)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:58.798034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:58.989624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b26af5c87b74a09b6fe8132e6b90452'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:58.993514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:59.054713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b26af5c87b74a09b6fe8132e6b90452'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.055346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b26af5c87b74a09b6fe8132e6b90452'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.055960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b26af5c87b74a09b6fe8132e6b90452'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.063819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-53985f8d-2d78-45c0-92be-ee713b71f8de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b26af5c87b74a09b6fe8132e6b90452'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.083001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1010/2022] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:58 2022] DELETE /identity/v3/users/1b26af5c87b74a09b6fe8132e6b90452 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:59.539977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:59.541569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:59.557761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358254e6-0dcf-4643-b848-27dd70b34b0c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:59.615670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:59.616309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:59.616584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:59.616854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] PATH_INFO: `/v3/projects/b30cf562d973440292978310ff1711da` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:59.617947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30cf562d973440292978310ff1711da)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:59.619947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:59.734276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30cf562d973440292978310ff1711da', 'request_id': 'req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.741899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30cf562d973440292978310ff1711da', 'request_id': 'req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.824320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30cf562d973440292978310ff1711da', 'request_id': 'req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.835506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] The token cache is being invalidate because project b30cf562d973440292978310ff1711da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:59.835794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.836420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d4a61ca7-7719-48b2-a502-8a1a1ccaea13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:59.837924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1013/2023] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:59 2022] DELETE /identity/v3/projects/b30cf562d973440292978310ff1711da => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:59.844259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775276-dcaa-40a8-9252-a5374c917848 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:59.844730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775276-dcaa-40a8-9252-a5374c917848 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:59.844936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775276-dcaa-40a8-9252-a5374c917848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:59.867371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-35775276-dcaa-40a8-9252-a5374c917848 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:59.868593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-35775276-dcaa-40a8-9252-a5374c917848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:59.932501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1011/2024] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:59.949518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd1776-6f27-4fdb-b953-00bc51362886 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:59.953787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd1776-6f27-4fdb-b953-00bc51362886 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:59.954419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd1776-6f27-4fdb-b953-00bc51362886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:59.975024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-00dd1776-6f27-4fdb-b953-00bc51362886 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:59.976316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-00dd1776-6f27-4fdb-b953-00bc51362886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:59.988547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1014/2025] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:59.990121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35775276-dcaa-40a8-9252-a5374c917848 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:00.000209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35775276-dcaa-40a8-9252-a5374c917848 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:00.005480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:00.006255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.006713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.006840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.007460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:00.007746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:00.009412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:00.033653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1012/2026] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:55:00.037761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd1776-6f27-4fdb-b953-00bc51362886 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:00.048150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd1776-6f27-4fdb-b953-00bc51362886 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:00.053947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6011cd-9366-4e7c-a387-18987f49a6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:00.054655 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6011cd-9366-4e7c-a387-18987f49a6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.057893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6011cd-9366-4e7c-a387-18987f49a6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.057893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6011cd-9366-4e7c-a387-18987f49a6c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.057893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6011cd-9366-4e7c-a387-18987f49a6c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:00.058231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6011cd-9366-4e7c-a387-18987f49a6c6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:00.153785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1015/2027] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:00 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:00.161139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:00.168965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939a2a7-e08e-49d7-8b15-e02b02de987b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:00.230310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7877d-7942-427b-9237-7df331dcde80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:00.231063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7877d-7942-427b-9237-7df331dcde80 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.231666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7877d-7942-427b-9237-7df331dcde80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.232111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7877d-7942-427b-9237-7df331dcde80 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.232755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7877d-7942-427b-9237-7df331dcde80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:00.235352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7877d-7942-427b-9237-7df331dcde80 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:00.335402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1013/2028] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:00 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:00.339750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6011cd-9366-4e7c-a387-18987f49a6c6 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:00.351014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6011cd-9366-4e7c-a387-18987f49a6c6 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:00.355681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9d1c9-1fe6-4c5f-8202-aa073e30bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:00.356274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9d1c9-1fe6-4c5f-8202-aa073e30bde1 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.356551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9d1c9-1fe6-4c5f-8202-aa073e30bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.356818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9d1c9-1fe6-4c5f-8202-aa073e30bde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.357457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9d1c9-1fe6-4c5f-8202-aa073e30bde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:00.359082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9d1c9-1fe6-4c5f-8202-aa073e30bde1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:00.369938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1016/2029] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:00.374163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7877d-7942-427b-9237-7df331dcde80 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:00.385866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7877d-7942-427b-9237-7df331dcde80 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:00.391813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb1973-cb4f-4b2c-9dd9-d42ee5241f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:00.392427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb1973-cb4f-4b2c-9dd9-d42ee5241f73 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.392710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb1973-cb4f-4b2c-9dd9-d42ee5241f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.392982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb1973-cb4f-4b2c-9dd9-d42ee5241f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.394042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb1973-cb4f-4b2c-9dd9-d42ee5241f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:00.395508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb1973-cb4f-4b2c-9dd9-d42ee5241f73 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:00.410623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1014/2030] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:00.412231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9d1c9-1fe6-4c5f-8202-aa073e30bde1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:00.423149 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9d1c9-1fe6-4c5f-8202-aa073e30bde1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:00.428878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a2f42-2669-464f-9807-65622e450309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:00.429625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a2f42-2669-464f-9807-65622e450309 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.429981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a2f42-2669-464f-9807-65622e450309 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.430366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a2f42-2669-464f-9807-65622e450309 admin admin] PATH_INFO: `/v3/projects/36c83cdd4a5a4fb281d98f294b765a7a/users/f1efb93e72284c85a87c09f53cd92d89/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.447680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a2f42-2669-464f-9807-65622e450309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c83cdd4a5a4fb281d98f294b765a7a, user_id=f1efb93e72284c85a87c09f53cd92d89, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:00.450766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a2f42-2669-464f-9807-65622e450309 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:00.545497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1017/2031] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:00 2022] PUT /identity/v3/projects/36c83cdd4a5a4fb281d98f294b765a7a/users/f1efb93e72284c85a87c09f53cd92d89/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:00.557456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7f0ce-c2d4-4d20-b266-a7f0933ae7b5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.557746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7f0ce-c2d4-4d20-b266-a7f0933ae7b5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.558047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7f0ce-c2d4-4d20-b266-a7f0933ae7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.586768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-a2d7f0ce-c2d4-4d20-b266-a7f0933ae7b5 None None] MFA Rules not processed for user `f1efb93e72284c85a87c09f53cd92d89`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:00.587942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a2d7f0ce-c2d4-4d20-b266-a7f0933ae7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:00.614323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:00.656174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1015/2032] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:00.661415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a2f42-2669-464f-9807-65622e450309 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:00.667200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:00.725246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a2f42-2669-464f-9807-65622e450309 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:00.730083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4962e-18f7-4fe9-9ea5-db05b49e0cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:00.730837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4962e-18f7-4fe9-9ea5-db05b49e0cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.731128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4962e-18f7-4fe9-9ea5-db05b49e0cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.731407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4962e-18f7-4fe9-9ea5-db05b49e0cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.732048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4962e-18f7-4fe9-9ea5-db05b49e0cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:00.734025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4962e-18f7-4fe9-9ea5-db05b49e0cf8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:00.877404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1018/2033] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:00 2022] POST /identity/v3/projects => generated 381 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:00.881424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7f0ce-c2d4-4d20-b266-a7f0933ae7b5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:00.893899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7f0ce-c2d4-4d20-b266-a7f0933ae7b5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:00.952867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3d74e-639c-4e92-91cd-49f6e51c4672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:00.954083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d74e-639c-4e92-91cd-49f6e51c4672 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:00.954393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d74e-639c-4e92-91cd-49f6e51c4672 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:00.954669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d74e-639c-4e92-91cd-49f6e51c4672 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:00.955318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3d74e-639c-4e92-91cd-49f6e51c4672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:00.957319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3d74e-639c-4e92-91cd-49f6e51c4672 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:01.066921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1016/2034] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:00 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:01.070877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4962e-18f7-4fe9-9ea5-db05b49e0cf8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:01.082071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4962e-18f7-4fe9-9ea5-db05b49e0cf8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:01.086638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b13580-d1ea-46b5-84e5-cc040a52c75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:01.087233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13580-d1ea-46b5-84e5-cc040a52c75b admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.087511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13580-d1ea-46b5-84e5-cc040a52c75b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.087778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13580-d1ea-46b5-84e5-cc040a52c75b admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.088410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b13580-d1ea-46b5-84e5-cc040a52c75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:01.090000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b13580-d1ea-46b5-84e5-cc040a52c75b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:01.103539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1019/2035] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:01.104869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3d74e-639c-4e92-91cd-49f6e51c4672 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:01.116439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3d74e-639c-4e92-91cd-49f6e51c4672 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:01.121163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55280a-f72c-4bc4-b5d5-bb202f176300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:01.121782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55280a-f72c-4bc4-b5d5-bb202f176300 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.122065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55280a-f72c-4bc4-b5d5-bb202f176300 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.122377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55280a-f72c-4bc4-b5d5-bb202f176300 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.123026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55280a-f72c-4bc4-b5d5-bb202f176300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:01.124667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55280a-f72c-4bc4-b5d5-bb202f176300 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:01.135703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1017/2036] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:01.139766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b13580-d1ea-46b5-84e5-cc040a52c75b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:01.151116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b13580-d1ea-46b5-84e5-cc040a52c75b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:01.156010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c095ec6-7b1b-41c9-a250-e8e5101e4f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:01.156681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c095ec6-7b1b-41c9-a250-e8e5101e4f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.157025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c095ec6-7b1b-41c9-a250-e8e5101e4f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.157303 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c095ec6-7b1b-41c9-a250-e8e5101e4f4d admin admin] PATH_INFO: `/v3/projects/3cf1098cc6494c7aa9fa0e043d7b9f14/users/ef582107aa244aa5a32ddbcf44d3444a/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.173873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c095ec6-7b1b-41c9-a250-e8e5101e4f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf1098cc6494c7aa9fa0e043d7b9f14, user_id=ef582107aa244aa5a32ddbcf44d3444a, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:01.178546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c095ec6-7b1b-41c9-a250-e8e5101e4f4d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:01.268359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1020/2037] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:01 2022] PUT /identity/v3/projects/3cf1098cc6494c7aa9fa0e043d7b9f14/users/ef582107aa244aa5a32ddbcf44d3444a/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:01.278915 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d4dc-3cf7-4d90-84b9-1bd2c85ac452 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.279201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d4dc-3cf7-4d90-84b9-1bd2c85ac452 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.279472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d4dc-3cf7-4d90-84b9-1bd2c85ac452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.331531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5042d4dc-3cf7-4d90-84b9-1bd2c85ac452 None None] MFA Rules not processed for user `ef582107aa244aa5a32ddbcf44d3444a`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:01.335615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5042d4dc-3cf7-4d90-84b9-1bd2c85ac452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:01.428050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1018/2038] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:01.432858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c095ec6-7b1b-41c9-a250-e8e5101e4f4d admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:01.518452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c095ec6-7b1b-41c9-a250-e8e5101e4f4d admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:01.523307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a68909-2d45-4061-90d6-f6fe413af29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:01.523931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68909-2d45-4061-90d6-f6fe413af29d admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.524214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68909-2d45-4061-90d6-f6fe413af29d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.524492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68909-2d45-4061-90d6-f6fe413af29d admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.525149 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a68909-2d45-4061-90d6-f6fe413af29d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:01.527147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a68909-2d45-4061-90d6-f6fe413af29d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:01.684439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1021/2039] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:01 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:01.686047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042d4dc-3cf7-4d90-84b9-1bd2c85ac452 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:01.698900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042d4dc-3cf7-4d90-84b9-1bd2c85ac452 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:01.737591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:01.765340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97c603-6844-476e-acfd-48fd53cc617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:01.765993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c603-6844-476e-acfd-48fd53cc617e admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.766574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c603-6844-476e-acfd-48fd53cc617e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.769951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97c603-6844-476e-acfd-48fd53cc617e admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.769951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97c603-6844-476e-acfd-48fd53cc617e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:01.769951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97c603-6844-476e-acfd-48fd53cc617e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:01.880992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1019/2040] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:01 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:01.882501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a68909-2d45-4061-90d6-f6fe413af29d admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:01.894846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a68909-2d45-4061-90d6-f6fe413af29d admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:01.902007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3686-17c5-4742-a6fe-dbc8ac9a390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:01.902674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3686-17c5-4742-a6fe-dbc8ac9a390a admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.902956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3686-17c5-4742-a6fe-dbc8ac9a390a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.903259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3686-17c5-4742-a6fe-dbc8ac9a390a admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.904071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab3686-17c5-4742-a6fe-dbc8ac9a390a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:01.906115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab3686-17c5-4742-a6fe-dbc8ac9a390a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:01.917775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1022/2041] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:01.922866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97c603-6844-476e-acfd-48fd53cc617e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:01.932879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97c603-6844-476e-acfd-48fd53cc617e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:01.937981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb936274-aee7-41ca-afe6-70dbbda12b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:01.938733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb936274-aee7-41ca-afe6-70dbbda12b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.939042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb936274-aee7-41ca-afe6-70dbbda12b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.939316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb936274-aee7-41ca-afe6-70dbbda12b5c admin admin] PATH_INFO: `/v3/projects/d58c574ef2824ee497a75cf7d8276e5c/users/2333f82197a843cd9b80fb2f9c89273f/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.954863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb936274-aee7-41ca-afe6-70dbbda12b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58c574ef2824ee497a75cf7d8276e5c, user_id=2333f82197a843cd9b80fb2f9c89273f, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:01.957677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb936274-aee7-41ca-afe6-70dbbda12b5c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:02.073087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1020/2042] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:01 2022] PUT /identity/v3/projects/d58c574ef2824ee497a75cf7d8276e5c/users/2333f82197a843cd9b80fb2f9c89273f/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:02.077161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3686-17c5-4742-a6fe-dbc8ac9a390a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:02.088475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3686-17c5-4742-a6fe-dbc8ac9a390a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:02.128773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:02.165465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f88691-b28c-470e-9835-e8dd5e9f2915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:02.166155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88691-b28c-470e-9835-e8dd5e9f2915 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:02.166624 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88691-b28c-470e-9835-e8dd5e9f2915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:02.166913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88691-b28c-470e-9835-e8dd5e9f2915 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:02.167607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f88691-b28c-470e-9835-e8dd5e9f2915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:02.169613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f88691-b28c-470e-9835-e8dd5e9f2915 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:02.192879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1023/2043] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:02.197268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb936274-aee7-41ca-afe6-70dbbda12b5c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:02.208992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb936274-aee7-41ca-afe6-70dbbda12b5c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:02.214008 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:02.214776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:02.215341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:02.216014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] PATH_INFO: `/v3/projects/d58c574ef2824ee497a75cf7d8276e5c/users/2333f82197a843cd9b80fb2f9c89273f/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:02.218738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58c574ef2824ee497a75cf7d8276e5c, user_id=2333f82197a843cd9b80fb2f9c89273f, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:02.222129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:02.233821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1021/2044] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:02 2022] PUT /identity/v3/projects/d58c574ef2824ee497a75cf7d8276e5c/users/2333f82197a843cd9b80fb2f9c89273f/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:02.245099 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e72d1-8106-4a5a-ba76-628e7fde5ea4 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:02.245663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e72d1-8106-4a5a-ba76-628e7fde5ea4 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:02.246035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e72d1-8106-4a5a-ba76-628e7fde5ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:02.281083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-004e72d1-8106-4a5a-ba76-628e7fde5ea4 None None] MFA Rules not processed for user `2333f82197a843cd9b80fb2f9c89273f`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:02.285476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-004e72d1-8106-4a5a-ba76-628e7fde5ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:02.362465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1024/2045] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:02.372216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:02.374179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:02.441227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7b55f-6ddd-4fb3-a1e1-300bbec0612e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:02.446200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:02.446521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:02.446854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:02.447409 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:02.521185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:02.522703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:02.533766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1022/2046] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:07.522505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e72d1-8106-4a5a-ba76-628e7fde5ea4 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:07.531325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e72d1-8106-4a5a-ba76-628e7fde5ea4 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:07.535574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:07.536242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:07.536584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:07.536924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:07.539254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:07.608526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:07.609981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:07.620412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1025/2047] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:08.625921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:08.627529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:08.640767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56897-929d-4394-a054-69e02bbc03d1 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:08.645082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8c492-7074-4c01-85ad-e785051e114b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:08.645687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8c492-7074-4c01-85ad-e785051e114b service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:08.645972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8c492-7074-4c01-85ad-e785051e114b service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:08.646283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8c492-7074-4c01-85ad-e785051e114b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:08.687979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:08.705836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8c492-7074-4c01-85ad-e785051e114b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:08.707301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8c492-7074-4c01-85ad-e785051e114b service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:08.716860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1023/2048] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:09.179288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:09.180949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:09.229333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:09.248657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50775147-90bf-4a56-a202-5bb7a08bc3f5 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:09.253461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:09.254097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:09.254412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:09.254686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] PATH_INFO: `/v3/users/3f3054d09a3441509b26cf53714214b6` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:09.255821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3054d09a3441509b26cf53714214b6)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:09.257752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:09.418126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3054d09a3441509b26cf53714214b6'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:09.512443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3054d09a3441509b26cf53714214b6'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:09.513048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3054d09a3441509b26cf53714214b6'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:09.527584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3054d09a3441509b26cf53714214b6'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:09.535684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3054d09a3441509b26cf53714214b6'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:09.541949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1026/2049] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:09 2022] DELETE /identity/v3/users/3f3054d09a3441509b26cf53714214b6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:09.546636 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e9a62-9f62-4d9a-b439-ef20e5d23795 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:09.546949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e9a62-9f62-4d9a-b439-ef20e5d23795 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:09.547205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e9a62-9f62-4d9a-b439-ef20e5d23795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:09.566166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c23e9a62-9f62-4d9a-b439-ef20e5d23795 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:09.567431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c23e9a62-9f62-4d9a-b439-ef20e5d23795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:09.629464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1024/2050] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:09.639756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:09.641498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:09.701878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2329b3-7d83-4c0a-8a5f-ad70d9006663 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:09.705575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b1bbe-3dd5-42aa-ac51-167d77c0faeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:09.706195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b1bbe-3dd5-42aa-ac51-167d77c0faeb service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:09.706473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b1bbe-3dd5-42aa-ac51-167d77c0faeb service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:09.707243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b1bbe-3dd5-42aa-ac51-167d77c0faeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:09.717833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b1bbe-3dd5-42aa-ac51-167d77c0faeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:09.719311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b1bbe-3dd5-42aa-ac51-167d77c0faeb service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:09.729830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1027/2051] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:10.222322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e9a62-9f62-4d9a-b439-ef20e5d23795 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:10.223752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c23e9a62-9f62-4d9a-b439-ef20e5d23795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:10.239797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e9a62-9f62-4d9a-b439-ef20e5d23795 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:10.245225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:10.245957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:10.246195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:10.246542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] PATH_INFO: `/v3/projects/708847196d6440d5886ef0f6d77e7ddc` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:10.247751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=708847196d6440d5886ef0f6d77e7ddc)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:10.249700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:10.369111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '708847196d6440d5886ef0f6d77e7ddc', 'request_id': 'req-23aa29a6-2160-4b23-b9e9-f14319d40bf9'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.375553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '708847196d6440d5886ef0f6d77e7ddc', 'request_id': 'req-23aa29a6-2160-4b23-b9e9-f14319d40bf9'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.443445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '708847196d6440d5886ef0f6d77e7ddc', 'request_id': 'req-23aa29a6-2160-4b23-b9e9-f14319d40bf9'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.454222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] The token cache is being invalidate because project 708847196d6440d5886ef0f6d77e7ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:10.454517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.455190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.456820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1025/2052] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:10 2022] DELETE /identity/v3/projects/708847196d6440d5886ef0f6d77e7ddc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:10.460836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b1bbe-3dd5-42aa-ac51-167d77c0faeb service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:10.470913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b1bbe-3dd5-42aa-ac51-167d77c0faeb service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:10.531915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:10.532550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:10.532835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:10.533115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] PATH_INFO: `/v3/users/259683c782a1439ab402bdc3bf7caed8` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:10.534283 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=259683c782a1439ab402bdc3bf7caed8)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:10.536253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:10.542412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:10.698337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '259683c782a1439ab402bdc3bf7caed8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.758021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '259683c782a1439ab402bdc3bf7caed8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.758680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '259683c782a1439ab402bdc3bf7caed8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.771469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '259683c782a1439ab402bdc3bf7caed8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.778829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b861fe40-182b-4910-b7de-1f80a7754088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '259683c782a1439ab402bdc3bf7caed8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:10.785429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1028/2053] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:10 2022] DELETE /identity/v3/users/259683c782a1439ab402bdc3bf7caed8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:11.254068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:11.256192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:11.274300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa29a6-2160-4b23-b9e9-f14319d40bf9 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:11.335524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:11.336170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:11.336448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:11.336723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] PATH_INFO: `/v3/projects/4086e880f09a4f71a3a44efa4a5611bb` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:11.337854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4086e880f09a4f71a3a44efa4a5611bb)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:11.339844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:11.342302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:11.456592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4086e880f09a4f71a3a44efa4a5611bb', 'request_id': 'req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:11.463971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4086e880f09a4f71a3a44efa4a5611bb', 'request_id': 'req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:11.552899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4086e880f09a4f71a3a44efa4a5611bb', 'request_id': 'req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:11.567911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] The token cache is being invalidate because project 4086e880f09a4f71a3a44efa4a5611bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:11.568206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:11.568883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1e5e3fcd-c69d-47e7-9b23-fa81bc919dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:11.572270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1026/2054] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:11 2022] DELETE /identity/v3/projects/4086e880f09a4f71a3a44efa4a5611bb => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:11.579538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540f10-74f8-4c9a-82ec-738748dddfec None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:11.580044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540f10-74f8-4c9a-82ec-738748dddfec None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:11.580184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540f10-74f8-4c9a-82ec-738748dddfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:11.601051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-26540f10-74f8-4c9a-82ec-738748dddfec None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:11.602300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-26540f10-74f8-4c9a-82ec-738748dddfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:11.689015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1029/2055] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:11.697511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee46e-a087-41d8-bb9b-7d6e7595bc25 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:11.697932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee46e-a087-41d8-bb9b-7d6e7595bc25 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:11.698234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee46e-a087-41d8-bb9b-7d6e7595bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:11.719668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5f3ee46e-a087-41d8-bb9b-7d6e7595bc25 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:11.720933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5f3ee46e-a087-41d8-bb9b-7d6e7595bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:11.729777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1027/2056] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:11.734127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26540f10-74f8-4c9a-82ec-738748dddfec None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:11.745368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26540f10-74f8-4c9a-82ec-738748dddfec None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:11.752717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:11.753402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:11.753714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:11.754547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:11.755251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:11.755559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:11.757176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:11.769636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1030/2057] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:55:11.774187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ee46e-a087-41d8-bb9b-7d6e7595bc25 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:11.787839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ee46e-a087-41d8-bb9b-7d6e7595bc25 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:11.792530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f6048-6f32-4903-af12-f07c913368e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:11.793630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f6048-6f32-4903-af12-f07c913368e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:11.794219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f6048-6f32-4903-af12-f07c913368e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:11.794925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f6048-6f32-4903-af12-f07c913368e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:11.795777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f6048-6f32-4903-af12-f07c913368e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:11.801065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f6048-6f32-4903-af12-f07c913368e7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:11.937287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1028/2058] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:11 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:11.941793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:11.952872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b98b8-a3e5-42b4-afa6-4c9dc6c91bef admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.010750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92207-f27e-40ba-a494-971ff94098bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.011351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92207-f27e-40ba-a494-971ff94098bf admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.011629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92207-f27e-40ba-a494-971ff94098bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.011900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92207-f27e-40ba-a494-971ff94098bf admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.012526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b92207-f27e-40ba-a494-971ff94098bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.014514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b92207-f27e-40ba-a494-971ff94098bf admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.130688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1031/2059] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:11 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:12.134420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f6048-6f32-4903-af12-f07c913368e7 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:12.149426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f6048-6f32-4903-af12-f07c913368e7 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.156545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133ffd2-53d0-43a6-b1bf-cd126fa26272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.157443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133ffd2-53d0-43a6-b1bf-cd126fa26272 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.157874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133ffd2-53d0-43a6-b1bf-cd126fa26272 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.158360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133ffd2-53d0-43a6-b1bf-cd126fa26272 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.159445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133ffd2-53d0-43a6-b1bf-cd126fa26272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.167822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133ffd2-53d0-43a6-b1bf-cd126fa26272 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.182855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1029/2060] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:12.187668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92207-f27e-40ba-a494-971ff94098bf admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:12.196054 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:12.203380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92207-f27e-40ba-a494-971ff94098bf admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.208037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea262676-e522-4914-9f8a-f63889c5da9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.208635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262676-e522-4914-9f8a-f63889c5da9b admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.209566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262676-e522-4914-9f8a-f63889c5da9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.209847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262676-e522-4914-9f8a-f63889c5da9b admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.210544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea262676-e522-4914-9f8a-f63889c5da9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.212195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea262676-e522-4914-9f8a-f63889c5da9b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.237346 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1032/2061] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:12.244311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133ffd2-53d0-43a6-b1bf-cd126fa26272 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:12.269098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133ffd2-53d0-43a6-b1bf-cd126fa26272 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.269098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd759295-6c4c-4e0e-bb08-60357954e720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.269098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759295-6c4c-4e0e-bb08-60357954e720 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.269098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759295-6c4c-4e0e-bb08-60357954e720 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.269977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759295-6c4c-4e0e-bb08-60357954e720 admin admin] PATH_INFO: `/v3/projects/36a20625c06b49febf8ab6d693ddf235/users/ad01bce065284993b1a72e7193ee9b18/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.286683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd759295-6c4c-4e0e-bb08-60357954e720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a20625c06b49febf8ab6d693ddf235, user_id=ad01bce065284993b1a72e7193ee9b18, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.290861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd759295-6c4c-4e0e-bb08-60357954e720 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.393293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1030/2062] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:12 2022] PUT /identity/v3/projects/36a20625c06b49febf8ab6d693ddf235/users/ad01bce065284993b1a72e7193ee9b18/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:12.403764 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb69ca-796c-4603-9343-757c91bbc493 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.404043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb69ca-796c-4603-9343-757c91bbc493 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.404310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb69ca-796c-4603-9343-757c91bbc493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.433890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e0cb69ca-796c-4603-9343-757c91bbc493 None None] MFA Rules not processed for user `ad01bce065284993b1a72e7193ee9b18`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:12.435238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e0cb69ca-796c-4603-9343-757c91bbc493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:12.502411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1033/2063] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:12.507048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd759295-6c4c-4e0e-bb08-60357954e720 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:12.579753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd759295-6c4c-4e0e-bb08-60357954e720 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.588582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae620a9a-722e-4ec7-8288-dc632469d354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.589394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae620a9a-722e-4ec7-8288-dc632469d354 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.590443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae620a9a-722e-4ec7-8288-dc632469d354 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.590791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae620a9a-722e-4ec7-8288-dc632469d354 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.591461 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae620a9a-722e-4ec7-8288-dc632469d354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.595020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae620a9a-722e-4ec7-8288-dc632469d354 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.623997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb69ca-796c-4603-9343-757c91bbc493 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:12.626766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e0cb69ca-796c-4603-9343-757c91bbc493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:12.687132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb69ca-796c-4603-9343-757c91bbc493 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.691505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.692130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.692411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.692681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.694517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:12.700971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1031/2064] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:12 2022] POST /identity/v3/projects => generated 369 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:12.702054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae620a9a-722e-4ec7-8288-dc632469d354 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:12.716332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:12.764257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.765872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.768783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae620a9a-722e-4ec7-8288-dc632469d354 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.774102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eea529-c1fc-40bd-ba13-55cc4689a91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.775605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eea529-c1fc-40bd-ba13-55cc4689a91a admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.775605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eea529-c1fc-40bd-ba13-55cc4689a91a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.775605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eea529-c1fc-40bd-ba13-55cc4689a91a admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.776066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eea529-c1fc-40bd-ba13-55cc4689a91a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.776698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1034/2065] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:12.779724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eea529-c1fc-40bd-ba13-55cc4689a91a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.881989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1032/2066] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:12 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:12.883064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:12.893850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3477f4-8ca5-4255-87de-c8ec49a97f9c service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.899896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d627c-eb27-4b4d-aaf5-0daa3de5d09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.900501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d627c-eb27-4b4d-aaf5-0daa3de5d09b admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.900915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d627c-eb27-4b4d-aaf5-0daa3de5d09b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.901202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d627c-eb27-4b4d-aaf5-0daa3de5d09b admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.901879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d627c-eb27-4b4d-aaf5-0daa3de5d09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.903537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d627c-eb27-4b4d-aaf5-0daa3de5d09b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.917913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1035/2067] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:12.919107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eea529-c1fc-40bd-ba13-55cc4689a91a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:12.929293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eea529-c1fc-40bd-ba13-55cc4689a91a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:12.934041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342108d-ae23-4d48-96f6-c794bfedb553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:12.934730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342108d-ae23-4d48-96f6-c794bfedb553 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:12.935014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342108d-ae23-4d48-96f6-c794bfedb553 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:12.935290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342108d-ae23-4d48-96f6-c794bfedb553 admin admin] PATH_INFO: `/v3/projects/db59ff9bfc4247108183b3a713c227f0/users/fc1e50ee915048998519eb4268ac7b16/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:12.951207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342108d-ae23-4d48-96f6-c794bfedb553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db59ff9bfc4247108183b3a713c227f0, user_id=fc1e50ee915048998519eb4268ac7b16, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:12.953831 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342108d-ae23-4d48-96f6-c794bfedb553 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:13.043348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1033/2068] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:12 2022] PUT /identity/v3/projects/db59ff9bfc4247108183b3a713c227f0/users/fc1e50ee915048998519eb4268ac7b16/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:13.047319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d627c-eb27-4b4d-aaf5-0daa3de5d09b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:13.058253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d627c-eb27-4b4d-aaf5-0daa3de5d09b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:13.075869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:13.116664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad8e86-1fb1-49b6-ac25-17d09969ba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:13.117269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad8e86-1fb1-49b6-ac25-17d09969ba6d admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:13.117553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad8e86-1fb1-49b6-ac25-17d09969ba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:13.119137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad8e86-1fb1-49b6-ac25-17d09969ba6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:13.119137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad8e86-1fb1-49b6-ac25-17d09969ba6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:13.120130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad8e86-1fb1-49b6-ac25-17d09969ba6d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:13.130273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1036/2069] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:13.134899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342108d-ae23-4d48-96f6-c794bfedb553 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:13.147491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342108d-ae23-4d48-96f6-c794bfedb553 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:13.152110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:13.152758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:13.153035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:13.153306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] PATH_INFO: `/v3/projects/db59ff9bfc4247108183b3a713c227f0/users/fc1e50ee915048998519eb4268ac7b16/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:13.155372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db59ff9bfc4247108183b3a713c227f0, user_id=fc1e50ee915048998519eb4268ac7b16, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:13.158010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:13.168036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1034/2070] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:13 2022] PUT /identity/v3/projects/db59ff9bfc4247108183b3a713c227f0/users/fc1e50ee915048998519eb4268ac7b16/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:13.179207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c45cb-ed10-4779-8f39-031044b41069 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:13.182292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c45cb-ed10-4779-8f39-031044b41069 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:13.182292 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c45cb-ed10-4779-8f39-031044b41069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:13.208568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-211c45cb-ed10-4779-8f39-031044b41069 None None] MFA Rules not processed for user `fc1e50ee915048998519eb4268ac7b16`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:13.209720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-211c45cb-ed10-4779-8f39-031044b41069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:13.278559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1037/2071] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:13.291024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:13.293084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:13.355746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f383b-a241-4a17-b2cf-d5b7cdc84e01 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:13.359538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a895ea7-51f4-4f24-8917-e741b06ff12b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:13.360147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a895ea7-51f4-4f24-8917-e741b06ff12b service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:13.360431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a895ea7-51f4-4f24-8917-e741b06ff12b service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:13.360708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a895ea7-51f4-4f24-8917-e741b06ff12b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:13.370078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a895ea7-51f4-4f24-8917-e741b06ff12b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:13.371524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a895ea7-51f4-4f24-8917-e741b06ff12b service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:13.381912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1035/2072] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:13.515394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c45cb-ed10-4779-8f39-031044b41069 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:13.525274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c45cb-ed10-4779-8f39-031044b41069 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:13.528970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc507c6-4022-4c91-98d7-972a9c2329d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:13.529564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc507c6-4022-4c91-98d7-972a9c2329d4 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:13.529845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc507c6-4022-4c91-98d7-972a9c2329d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:13.530119 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc507c6-4022-4c91-98d7-972a9c2329d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:13.593120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc507c6-4022-4c91-98d7-972a9c2329d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:13.594595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc507c6-4022-4c91-98d7-972a9c2329d4 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:13.604632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1038/2073] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:19.054742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a895ea7-51f4-4f24-8917-e741b06ff12b service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:19.165285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a895ea7-51f4-4f24-8917-e741b06ff12b service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:19.171769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:19.172637 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:19.172997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:19.173893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] PATH_INFO: `/v3/users/ad01bce065284993b1a72e7193ee9b18` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:19.175235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad01bce065284993b1a72e7193ee9b18)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:19.177778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:19.196134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:19.407863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad01bce065284993b1a72e7193ee9b18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:19.473195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad01bce065284993b1a72e7193ee9b18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:19.473900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad01bce065284993b1a72e7193ee9b18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:19.474925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad01bce065284993b1a72e7193ee9b18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:19.482687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad01bce065284993b1a72e7193ee9b18'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:19.505227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1036/2074] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:19 2022] DELETE /identity/v3/users/ad01bce065284993b1a72e7193ee9b18 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:19.509573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4619012-ca51-4c6d-957c-e89a1cf4e51b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:19.510037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4619012-ca51-4c6d-957c-e89a1cf4e51b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:19.510272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4619012-ca51-4c6d-957c-e89a1cf4e51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:19.530763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c4619012-ca51-4c6d-957c-e89a1cf4e51b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:19.532248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c4619012-ca51-4c6d-957c-e89a1cf4e51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:19.569231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:19.596349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1039/2075] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:19.606750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:19.608337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:19.690432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a447f-e986-486c-981a-b6ad2d283505 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:19.695972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbce6e-bd52-43b7-b1f8-f03fc69c8863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:19.695972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbce6e-bd52-43b7-b1f8-f03fc69c8863 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:19.695972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbce6e-bd52-43b7-b1f8-f03fc69c8863 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:19.696402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbce6e-bd52-43b7-b1f8-f03fc69c8863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:19.706426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbce6e-bd52-43b7-b1f8-f03fc69c8863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:19.708648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbce6e-bd52-43b7-b1f8-f03fc69c8863 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:19.724632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1037/2076] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:20.350648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4619012-ca51-4c6d-957c-e89a1cf4e51b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:20.353096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c4619012-ca51-4c6d-957c-e89a1cf4e51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:20.367608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4619012-ca51-4c6d-957c-e89a1cf4e51b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:20.377109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:20.381410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:20.381410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:20.381410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] PATH_INFO: `/v3/projects/36a20625c06b49febf8ab6d693ddf235` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:20.381410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a20625c06b49febf8ab6d693ddf235)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:20.382411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:20.543114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a20625c06b49febf8ab6d693ddf235', 'request_id': 'req-9840fa6e-7398-4f52-9f57-1f3d69622781'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:20.550374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a20625c06b49febf8ab6d693ddf235', 'request_id': 'req-9840fa6e-7398-4f52-9f57-1f3d69622781'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:20.604085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a20625c06b49febf8ab6d693ddf235', 'request_id': 'req-9840fa6e-7398-4f52-9f57-1f3d69622781'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:20.616890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] The token cache is being invalidate because project 36a20625c06b49febf8ab6d693ddf235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:20.617227 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:20.617902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:20.622168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1040/2077] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:20 2022] DELETE /identity/v3/projects/36a20625c06b49febf8ab6d693ddf235 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:20.624103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbce6e-bd52-43b7-b1f8-f03fc69c8863 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:20.640231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbce6e-bd52-43b7-b1f8-f03fc69c8863 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:20.711156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:20.711912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:20.712291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:20.712640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] PATH_INFO: `/v3/users/fc1e50ee915048998519eb4268ac7b16` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:20.713948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1e50ee915048998519eb4268ac7b16)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:20.716056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:20.897407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:20.911630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1e50ee915048998519eb4268ac7b16'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.004376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1e50ee915048998519eb4268ac7b16'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.004990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1e50ee915048998519eb4268ac7b16'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.005611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1e50ee915048998519eb4268ac7b16'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.015122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-00e09be3-9602-411a-8e7c-b2c875cf72a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1e50ee915048998519eb4268ac7b16'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.035687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1038/2078] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:20 2022] DELETE /identity/v3/users/fc1e50ee915048998519eb4268ac7b16 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:21.521343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:21.523268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:21.540224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840fa6e-7398-4f52-9f57-1f3d69622781 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:21.647058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:21.647704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:21.648100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:21.648276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] PATH_INFO: `/v3/projects/db59ff9bfc4247108183b3a713c227f0` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:21.649878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] RBAC: Authorizing `identity:delete_project(project_id=db59ff9bfc4247108183b3a713c227f0)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:21.651944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:21.802525 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db59ff9bfc4247108183b3a713c227f0', 'request_id': 'req-96998ad2-9111-4a98-bfae-b5263619a9df'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.811419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db59ff9bfc4247108183b3a713c227f0', 'request_id': 'req-96998ad2-9111-4a98-bfae-b5263619a9df'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.881300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db59ff9bfc4247108183b3a713c227f0', 'request_id': 'req-96998ad2-9111-4a98-bfae-b5263619a9df'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.905162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:21.916790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] The token cache is being invalidate because project db59ff9bfc4247108183b3a713c227f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:21.917137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.917869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-96998ad2-9111-4a98-bfae-b5263619a9df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:21.922648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1041/2079] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:21 2022] DELETE /identity/v3/projects/db59ff9bfc4247108183b3a713c227f0 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:21.925145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98376e-8c9a-4ef1-83e4-67207aff6895 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:21.925619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98376e-8c9a-4ef1-83e4-67207aff6895 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:21.925930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98376e-8c9a-4ef1-83e4-67207aff6895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:21.971457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5f98376e-8c9a-4ef1-83e4-67207aff6895 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:21.973316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5f98376e-8c9a-4ef1-83e4-67207aff6895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:22.072939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1039/2080] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:22.082582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5f79c-1b4b-4dfe-b394-bd17494e6844 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.083616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5f79c-1b4b-4dfe-b394-bd17494e6844 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.087136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5f79c-1b4b-4dfe-b394-bd17494e6844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.120648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c6e5f79c-1b4b-4dfe-b394-bd17494e6844 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:22.122101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c6e5f79c-1b4b-4dfe-b394-bd17494e6844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:22.133097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1042/2081] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:22.137238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98376e-8c9a-4ef1-83e4-67207aff6895 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.148884 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98376e-8c9a-4ef1-83e4-67207aff6895 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:22.154015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:22.154665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.156257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.156257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.156501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:22.156629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.158280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.171541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1040/2082] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:55:22.176188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5f79c-1b4b-4dfe-b394-bd17494e6844 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.186747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5f79c-1b4b-4dfe-b394-bd17494e6844 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:22.197506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595ef78-c436-47ef-be00-3f1fb5987e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:22.197506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595ef78-c436-47ef-be00-3f1fb5987e16 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.197506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595ef78-c436-47ef-be00-3f1fb5987e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.197506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595ef78-c436-47ef-be00-3f1fb5987e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.198176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595ef78-c436-47ef-be00-3f1fb5987e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.198176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595ef78-c436-47ef-be00-3f1fb5987e16 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.344654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1043/2083] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:22 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:22.345885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.365465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f06ae-79a9-4e75-98f8-831952ecf6a8 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:22.461779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934919e-948a-47dc-9f6e-9dbdaf4d913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:22.463172 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934919e-948a-47dc-9f6e-9dbdaf4d913d admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.463457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934919e-948a-47dc-9f6e-9dbdaf4d913d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.463734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934919e-948a-47dc-9f6e-9dbdaf4d913d admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.464520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934919e-948a-47dc-9f6e-9dbdaf4d913d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.469781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934919e-948a-47dc-9f6e-9dbdaf4d913d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.553738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1041/2084] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:22 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:22.557986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595ef78-c436-47ef-be00-3f1fb5987e16 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.569994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595ef78-c436-47ef-be00-3f1fb5987e16 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:22.574916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255b8dd-d564-46cf-ad6d-1266ea847959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:22.575542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255b8dd-d564-46cf-ad6d-1266ea847959 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.575826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255b8dd-d564-46cf-ad6d-1266ea847959 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.576096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255b8dd-d564-46cf-ad6d-1266ea847959 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.576733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255b8dd-d564-46cf-ad6d-1266ea847959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.578356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255b8dd-d564-46cf-ad6d-1266ea847959 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.588786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1044/2085] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:22.593010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934919e-948a-47dc-9f6e-9dbdaf4d913d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.604438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934919e-948a-47dc-9f6e-9dbdaf4d913d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:22.611034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a512173-7fcb-4989-8e80-592da7e2cb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:22.618320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a512173-7fcb-4989-8e80-592da7e2cb07 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.618320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a512173-7fcb-4989-8e80-592da7e2cb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.618320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a512173-7fcb-4989-8e80-592da7e2cb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.618320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a512173-7fcb-4989-8e80-592da7e2cb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.618320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a512173-7fcb-4989-8e80-592da7e2cb07 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.626873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1042/2086] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:22.631409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255b8dd-d564-46cf-ad6d-1266ea847959 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.643299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255b8dd-d564-46cf-ad6d-1266ea847959 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:22.648201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fe84a-0843-4260-af8b-a3f6a092259f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:22.648931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe84a-0843-4260-af8b-a3f6a092259f admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.649258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe84a-0843-4260-af8b-a3f6a092259f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.649519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe84a-0843-4260-af8b-a3f6a092259f admin admin] PATH_INFO: `/v3/projects/0f3172b5d47145b7bb46df4ce131dcf2/users/764a2584f5544c9d93ba471e72bd36ce/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.666401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fe84a-0843-4260-af8b-a3f6a092259f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3172b5d47145b7bb46df4ce131dcf2, user_id=764a2584f5544c9d93ba471e72bd36ce, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.669124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fe84a-0843-4260-af8b-a3f6a092259f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.754644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1045/2087] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:22 2022] PUT /identity/v3/projects/0f3172b5d47145b7bb46df4ce131dcf2/users/764a2584f5544c9d93ba471e72bd36ce/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:22.759044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ecee1-f7e1-434e-bdd1-60092782060e None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.759044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ecee1-f7e1-434e-bdd1-60092782060e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.759044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ecee1-f7e1-434e-bdd1-60092782060e None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.759044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1043/2088] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:22.760258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2abcd-7dd9-4fc9-8b06-4b6fdf1afc5b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.760549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2abcd-7dd9-4fc9-8b06-4b6fdf1afc5b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.760824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2abcd-7dd9-4fc9-8b06-4b6fdf1afc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.783063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-87e2abcd-7dd9-4fc9-8b06-4b6fdf1afc5b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:22.784627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-87e2abcd-7dd9-4fc9-8b06-4b6fdf1afc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:22.793280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267d0dc-9b28-4471-be53-2d7a05ef4f26 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.793752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267d0dc-9b28-4471-be53-2d7a05ef4f26 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.794233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267d0dc-9b28-4471-be53-2d7a05ef4f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.827264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3267d0dc-9b28-4471-be53-2d7a05ef4f26 None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:22.828741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3267d0dc-9b28-4471-be53-2d7a05ef4f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:22.839602 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:22.864838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1046/2089] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:22.877630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2abcd-7dd9-4fc9-8b06-4b6fdf1afc5b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.879267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-87e2abcd-7dd9-4fc9-8b06-4b6fdf1afc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:22.925529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1044/2090] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:55:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:22.926995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267d0dc-9b28-4471-be53-2d7a05ef4f26 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.928278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3267d0dc-9b28-4471-be53-2d7a05ef4f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:22.943090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267d0dc-9b28-4471-be53-2d7a05ef4f26 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:22.947991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:22.951242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.951242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.951242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] PATH_INFO: `/v3/users/8294dcc1178045abbcf7a497564eb122` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.951242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8294dcc1178045abbcf7a497564eb122)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.952618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.956513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2abcd-7dd9-4fc9-8b06-4b6fdf1afc5b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:22.961067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:22.961750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:22.962097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:22.962408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:22.973727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.975327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.986160 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1047/2091] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:22.986994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:22.988975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:23.003029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545f72c-0f4e-4b4b-9755-72a03d79f6c3 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:23.008545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c1c62-0278-49a2-827d-ffc9d7429a01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:23.009153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c1c62-0278-49a2-827d-ffc9d7429a01 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:23.009426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c1c62-0278-49a2-827d-ffc9d7429a01 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:23.009697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c1c62-0278-49a2-827d-ffc9d7429a01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:23.022785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c1c62-0278-49a2-827d-ffc9d7429a01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:23.024352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c1c62-0278-49a2-827d-ffc9d7429a01 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:23.034566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1048/2092] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:23.183768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8294dcc1178045abbcf7a497564eb122'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:23.258833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8294dcc1178045abbcf7a497564eb122'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:23.259757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8294dcc1178045abbcf7a497564eb122'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:23.260569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8294dcc1178045abbcf7a497564eb122'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:23.278392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02169a02-c8c7-4d4b-9d08-c829a067b1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8294dcc1178045abbcf7a497564eb122'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:23.303218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1045/2093] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:22 2022] DELETE /identity/v3/users/8294dcc1178045abbcf7a497564eb122 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:23.663393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c1c62-0278-49a2-827d-ffc9d7429a01 service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:23.675390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c1c62-0278-49a2-827d-ffc9d7429a01 service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:23.713004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:23.756767 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:23.757400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:23.757678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:23.757953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] PATH_INFO: `/v3/projects/5775050045e749b1bdb26dddcce9a8ff` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:23.759100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5775050045e749b1bdb26dddcce9a8ff)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:23.761002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:23.922459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5775050045e749b1bdb26dddcce9a8ff', 'request_id': 'req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:23.932251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5775050045e749b1bdb26dddcce9a8ff', 'request_id': 'req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.037376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5775050045e749b1bdb26dddcce9a8ff', 'request_id': 'req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.088109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] The token cache is being invalidate because project 5775050045e749b1bdb26dddcce9a8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:24.088109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.091520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.093710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1049/2094] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:23 2022] DELETE /identity/v3/projects/5775050045e749b1bdb26dddcce9a8ff => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:24.105773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b43f0-3c5b-453b-8dc6-7138728bb628 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.110662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b43f0-3c5b-453b-8dc6-7138728bb628 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.110662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b43f0-3c5b-453b-8dc6-7138728bb628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.111006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:24.112286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.131334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-db3b43f0-3c5b-453b-8dc6-7138728bb628 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:24.133090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-db3b43f0-3c5b-453b-8dc6-7138728bb628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.191490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc90b4a-21b1-4ae4-a823-4c195eeb2727 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:24.196553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:24.197303 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.197592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.198015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] PATH_INFO: `/v3/users/f1efb93e72284c85a87c09f53cd92d89` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.199223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1efb93e72284c85a87c09f53cd92d89)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:24.201276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:24.210743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1046/2095] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:24.219585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899c54d-f7a0-4f57-b6a9-3e83eb6cada5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.219943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899c54d-f7a0-4f57-b6a9-3e83eb6cada5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.220294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899c54d-f7a0-4f57-b6a9-3e83eb6cada5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.242317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b899c54d-f7a0-4f57-b6a9-3e83eb6cada5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:24.243600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b899c54d-f7a0-4f57-b6a9-3e83eb6cada5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.251824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1047/2096] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:24.256042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899c54d-f7a0-4f57-b6a9-3e83eb6cada5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:24.265782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899c54d-f7a0-4f57-b6a9-3e83eb6cada5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:24.271087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:24.271846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.272718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.272718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.273201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:24.273500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:24.275192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:24.292069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1048/2097] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:55:24.296499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:24.303759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:24.307246 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a5a0a-6b3d-4bb2-b2ae-f5801fef917b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:24.312391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9f6e8-e0bc-46df-b2a9-ad8ac6188498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:24.313004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9f6e8-e0bc-46df-b2a9-ad8ac6188498 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.313377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9f6e8-e0bc-46df-b2a9-ad8ac6188498 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.313656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9f6e8-e0bc-46df-b2a9-ad8ac6188498 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.314334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9f6e8-e0bc-46df-b2a9-ad8ac6188498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:24.316433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9f6e8-e0bc-46df-b2a9-ad8ac6188498 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:24.457644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1049/2098] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:24 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:24.459020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379f74-7d5b-43a0-b7ce-71f319b0444e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.459266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379f74-7d5b-43a0-b7ce-71f319b0444e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.459681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379f74-7d5b-43a0-b7ce-71f319b0444e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.468608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1efb93e72284c85a87c09f53cd92d89'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.473475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:24.497671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f4379f74-7d5b-43a0-b7ce-71f319b0444e None None] MFA Rules not processed for user `764a2584f5544c9d93ba471e72bd36ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:24.498963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f4379f74-7d5b-43a0-b7ce-71f319b0444e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.559632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1efb93e72284c85a87c09f53cd92d89'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.564617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1efb93e72284c85a87c09f53cd92d89'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.585886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1efb93e72284c85a87c09f53cd92d89'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.606599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ea54a2f0-0d7a-461c-bf0e-ff9b4e844410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1efb93e72284c85a87c09f53cd92d89'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:24.618718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1050/2099] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:24.618718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4379f74-7d5b-43a0-b7ce-71f319b0444e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:24.618718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f4379f74-7d5b-43a0-b7ce-71f319b0444e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.618987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1050/2100] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:24 2022] DELETE /identity/v3/users/f1efb93e72284c85a87c09f53cd92d89 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:24.621505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21edb3-d4a8-44c7-a71b-e087c7799d3c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.621796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21edb3-d4a8-44c7-a71b-e087c7799d3c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.622225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21edb3-d4a8-44c7-a71b-e087c7799d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.648637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-6d21edb3-d4a8-44c7-a71b-e087c7799d3c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:24.649792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6d21edb3-d4a8-44c7-a71b-e087c7799d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.721437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4379f74-7d5b-43a0-b7ce-71f319b0444e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:24.724151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1051/2101] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:24.725004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21edb3-d4a8-44c7-a71b-e087c7799d3c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:24.726374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6d21edb3-d4a8-44c7-a71b-e087c7799d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.731597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:24.732680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.733217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.733659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.734893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:24.742617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:24.742783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21edb3-d4a8-44c7-a71b-e087c7799d3c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:24.747526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe59ea-0003-441d-87ae-350ab4e1d556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:24.748252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe59ea-0003-441d-87ae-350ab4e1d556 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.748663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe59ea-0003-441d-87ae-350ab4e1d556 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.749037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe59ea-0003-441d-87ae-350ab4e1d556 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.749857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe59ea-0003-441d-87ae-350ab4e1d556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:24.752486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe59ea-0003-441d-87ae-350ab4e1d556 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:24.839678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1051/2102] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:24 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:24.840555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:24.843711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.907124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1052/2103] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:24 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:24.907899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe59ea-0003-441d-87ae-350ab4e1d556 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:24.911173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84899bea-1c08-47b8-8b11-0607696043b5 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:24.928815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:24.980563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366c364-3bdf-4b84-9d7e-b819a8ef3cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:24.981185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366c364-3bdf-4b84-9d7e-b819a8ef3cca service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.981493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366c364-3bdf-4b84-9d7e-b819a8ef3cca service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.981728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366c364-3bdf-4b84-9d7e-b819a8ef3cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.991669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe59ea-0003-441d-87ae-350ab4e1d556 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:24.996209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366c364-3bdf-4b84-9d7e-b819a8ef3cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:24.996481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1da25b-3e20-4257-831f-fdd1aa65f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:24.997083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da25b-3e20-4257-831f-fdd1aa65f36b admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.997422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da25b-3e20-4257-831f-fdd1aa65f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.997813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366c364-3bdf-4b84-9d7e-b819a8ef3cca service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:24.997958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da25b-3e20-4257-831f-fdd1aa65f36b admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.999145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1da25b-3e20-4257-831f-fdd1aa65f36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.001471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1da25b-3e20-4257-831f-fdd1aa65f36b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.009851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1052/2104] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:25.013768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1053/2105] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:25.015229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366c364-3bdf-4b84-9d7e-b819a8ef3cca service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.024433 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1da25b-3e20-4257-831f-fdd1aa65f36b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.028605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366c364-3bdf-4b84-9d7e-b819a8ef3cca service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.035334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73d60f-330b-4d86-a191-6ce65fdf013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.035968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d60f-330b-4d86-a191-6ce65fdf013b admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.036585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d60f-330b-4d86-a191-6ce65fdf013b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.036585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d60f-330b-4d86-a191-6ce65fdf013b admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.037169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73d60f-330b-4d86-a191-6ce65fdf013b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.037759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1da25b-3e20-4257-831f-fdd1aa65f36b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.039381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73d60f-330b-4d86-a191-6ce65fdf013b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.042595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef530fb-41d1-4ed1-a64c-a083c72995a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.043782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef530fb-41d1-4ed1-a64c-a083c72995a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.043782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef530fb-41d1-4ed1-a64c-a083c72995a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.043961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef530fb-41d1-4ed1-a64c-a083c72995a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.044485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef530fb-41d1-4ed1-a64c-a083c72995a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.046102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef530fb-41d1-4ed1-a64c-a083c72995a2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.060688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1054/2106] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:25.061833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef530fb-41d1-4ed1-a64c-a083c72995a2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.079172 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef530fb-41d1-4ed1-a64c-a083c72995a2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.087672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9ab4f-bb98-4358-b880-401dde0f813e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.088602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9ab4f-bb98-4358-b880-401dde0f813e admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.088883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9ab4f-bb98-4358-b880-401dde0f813e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.089191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9ab4f-bb98-4358-b880-401dde0f813e admin admin] PATH_INFO: `/v3/projects/d87aab0773ff4e4d9d90bd7d8f9943da/users/8026633a0f004a078a0daad083b78bfa/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.126016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9ab4f-bb98-4358-b880-401dde0f813e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87aab0773ff4e4d9d90bd7d8f9943da, user_id=8026633a0f004a078a0daad083b78bfa, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.130304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9ab4f-bb98-4358-b880-401dde0f813e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.163459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1053/2107] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:25 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:25.164713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73d60f-330b-4d86-a191-6ce65fdf013b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.175333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73d60f-330b-4d86-a191-6ce65fdf013b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.181630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbe174-2ab0-4e7d-b5fe-89fe25d87292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.181630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbe174-2ab0-4e7d-b5fe-89fe25d87292 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.181630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbe174-2ab0-4e7d-b5fe-89fe25d87292 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.182083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbe174-2ab0-4e7d-b5fe-89fe25d87292 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.182402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbe174-2ab0-4e7d-b5fe-89fe25d87292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.184510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbe174-2ab0-4e7d-b5fe-89fe25d87292 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.196064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1054/2108] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:25.200453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbe174-2ab0-4e7d-b5fe-89fe25d87292 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.211550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbe174-2ab0-4e7d-b5fe-89fe25d87292 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.216376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cb8d-730c-40ab-8bea-f3de584556ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.217061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cb8d-730c-40ab-8bea-f3de584556ea admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.217341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cb8d-730c-40ab-8bea-f3de584556ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.217618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cb8d-730c-40ab-8bea-f3de584556ea admin admin] PATH_INFO: `/v3/projects/417e2c4567c848b2937de6f1efbf3559/users/793674c65b4147b39b94bacb791e3286/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.233728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089cb8d-730c-40ab-8bea-f3de584556ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=417e2c4567c848b2937de6f1efbf3559, user_id=793674c65b4147b39b94bacb791e3286, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.236439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089cb8d-730c-40ab-8bea-f3de584556ea admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.250707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1055/2109] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:25 2022] PUT /identity/v3/projects/d87aab0773ff4e4d9d90bd7d8f9943da/users/8026633a0f004a078a0daad083b78bfa/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:25.261574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820451c-4b47-4fbb-b233-fd09f48500a0 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.261963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820451c-4b47-4fbb-b233-fd09f48500a0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.262353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820451c-4b47-4fbb-b233-fd09f48500a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.289912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:25.294526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d820451c-4b47-4fbb-b233-fd09f48500a0 None None] MFA Rules not processed for user `8026633a0f004a078a0daad083b78bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:25.295956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d820451c-4b47-4fbb-b233-fd09f48500a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:25.348252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1055/2110] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:25 2022] PUT /identity/v3/projects/417e2c4567c848b2937de6f1efbf3559/users/793674c65b4147b39b94bacb791e3286/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:25.352125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cb8d-730c-40ab-8bea-f3de584556ea admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.403692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1056/2111] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:25.408795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820451c-4b47-4fbb-b233-fd09f48500a0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.452570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cb8d-730c-40ab-8bea-f3de584556ea admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.457526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d022526-03d1-4d9a-accb-60f206e9c2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.458252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d022526-03d1-4d9a-accb-60f206e9c2ca admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.458960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d022526-03d1-4d9a-accb-60f206e9c2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.458960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d022526-03d1-4d9a-accb-60f206e9c2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.459780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d022526-03d1-4d9a-accb-60f206e9c2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.461512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d022526-03d1-4d9a-accb-60f206e9c2ca admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.472200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1056/2112] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:25.480231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d022526-03d1-4d9a-accb-60f206e9c2ca admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.485709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d022526-03d1-4d9a-accb-60f206e9c2ca admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.490920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.491560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.491843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.492119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] PATH_INFO: `/v3/projects/417e2c4567c848b2937de6f1efbf3559/users/793674c65b4147b39b94bacb791e3286/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.494056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] RBAC: Authorizing `identity:create_grant(project_id=417e2c4567c848b2937de6f1efbf3559, user_id=793674c65b4147b39b94bacb791e3286, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.496786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.497588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820451c-4b47-4fbb-b233-fd09f48500a0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.502608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c274-a491-446f-a0cb-cf2afed21dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.509016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c274-a491-446f-a0cb-cf2afed21dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.509016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c274-a491-446f-a0cb-cf2afed21dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.509016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c274-a491-446f-a0cb-cf2afed21dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.509016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08c274-a491-446f-a0cb-cf2afed21dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.509016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08c274-a491-446f-a0cb-cf2afed21dd8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.509626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1057/2113] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:25 2022] PUT /identity/v3/projects/417e2c4567c848b2937de6f1efbf3559/users/793674c65b4147b39b94bacb791e3286/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:25.572196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.577916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:25.620780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1057/2114] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:25 2022] POST /identity/v3/projects => generated 363 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:25.623704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c274-a491-446f-a0cb-cf2afed21dd8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.632313 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:25.647711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738daf8-1dd0-48ae-b7c5-5e7a1ce3a6db admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.698423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c274-a491-446f-a0cb-cf2afed21dd8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.703753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffc3d9-db8d-4dac-8fe8-df2ef9ceaf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.704407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffc3d9-db8d-4dac-8fe8-df2ef9ceaf7a admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.704646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffc3d9-db8d-4dac-8fe8-df2ef9ceaf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.704920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffc3d9-db8d-4dac-8fe8-df2ef9ceaf7a admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.705563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffc3d9-db8d-4dac-8fe8-df2ef9ceaf7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.707811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffc3d9-db8d-4dac-8fe8-df2ef9ceaf7a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.724234 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.724936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.725451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.725812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] PATH_INFO: `/v3/projects/36c83cdd4a5a4fb281d98f294b765a7a` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.727013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c83cdd4a5a4fb281d98f294b765a7a)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.729041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.803466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1058/2115] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:25 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:25.807492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffc3d9-db8d-4dac-8fe8-df2ef9ceaf7a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.817095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffc3d9-db8d-4dac-8fe8-df2ef9ceaf7a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.821963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2caf22-8370-433e-8657-4c27383fd5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.822595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2caf22-8370-433e-8657-4c27383fd5ec admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.822893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2caf22-8370-433e-8657-4c27383fd5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.823152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2caf22-8370-433e-8657-4c27383fd5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.823828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2caf22-8370-433e-8657-4c27383fd5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.825454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2caf22-8370-433e-8657-4c27383fd5ec admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.836389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1059/2116] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:25.849484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c83cdd4a5a4fb281d98f294b765a7a', 'request_id': 'req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:25.850974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2caf22-8370-433e-8657-4c27383fd5ec admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.856309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c83cdd4a5a4fb281d98f294b765a7a', 'request_id': 'req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:25.861180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2caf22-8370-433e-8657-4c27383fd5ec admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.921218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.921942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.922244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.922520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] PATH_INFO: `/v3/projects/617637240a5645b6b36e829c59e1ce55/users/5b977d2e68e44587af9b3d06f4323119/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.938587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=617637240a5645b6b36e829c59e1ce55, user_id=5b977d2e68e44587af9b3d06f4323119, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.942235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.946113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c83cdd4a5a4fb281d98f294b765a7a', 'request_id': 'req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:25.958612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] The token cache is being invalidate because project 36c83cdd4a5a4fb281d98f294b765a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:25.958869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:25.959729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:25.961871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1058/2117] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:25 2022] DELETE /identity/v3/projects/36c83cdd4a5a4fb281d98f294b765a7a => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:25.965493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.966944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:25.980495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c55ffc-1cc4-4c8c-a531-61257cd4fe6c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.985436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.986071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.986542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.986731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] PATH_INFO: `/v3/users/ef582107aa244aa5a32ddbcf44d3444a` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.987935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef582107aa244aa5a32ddbcf44d3444a)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.990553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:26.085193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1060/2118] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:25 2022] PUT /identity/v3/projects/617637240a5645b6b36e829c59e1ce55/users/5b977d2e68e44587af9b3d06f4323119/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:26.089052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:26.090634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:26.162497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4980e04-21ca-4b5f-b3f9-75511f13c1de admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:26.167216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c9959-ca10-4365-834a-1dd762508ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:26.167822 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c9959-ca10-4365-834a-1dd762508ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:26.168102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c9959-ca10-4365-834a-1dd762508ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:26.168939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c9959-ca10-4365-834a-1dd762508ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:26.169063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c9959-ca10-4365-834a-1dd762508ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:26.171393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c9959-ca10-4365-834a-1dd762508ab2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:26.174085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:26.187162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1061/2119] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:26.188779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c9959-ca10-4365-834a-1dd762508ab2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:26.198790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c9959-ca10-4365-834a-1dd762508ab2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:26.203613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81900e1-564c-4c0f-aba7-fdca9796ce5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:26.204268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81900e1-564c-4c0f-aba7-fdca9796ce5c admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:26.204550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81900e1-564c-4c0f-aba7-fdca9796ce5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:26.204831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81900e1-564c-4c0f-aba7-fdca9796ce5c admin admin] PATH_INFO: `/v3/projects/617637240a5645b6b36e829c59e1ce55/users/5b977d2e68e44587af9b3d06f4323119/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:26.206800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81900e1-564c-4c0f-aba7-fdca9796ce5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=617637240a5645b6b36e829c59e1ce55, user_id=5b977d2e68e44587af9b3d06f4323119, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:26.210097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81900e1-564c-4c0f-aba7-fdca9796ce5c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:26.220712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1062/2120] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:26 2022] PUT /identity/v3/projects/617637240a5645b6b36e829c59e1ce55/users/5b977d2e68e44587af9b3d06f4323119/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:26.235493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097176-3d80-436a-ac12-e2895d2befe9 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:26.235973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097176-3d80-436a-ac12-e2895d2befe9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:26.236388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097176-3d80-436a-ac12-e2895d2befe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:26.250829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef582107aa244aa5a32ddbcf44d3444a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:26.273691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-2c097176-3d80-436a-ac12-e2895d2befe9 None None] MFA Rules not processed for user `5b977d2e68e44587af9b3d06f4323119`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:26.274999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2c097176-3d80-436a-ac12-e2895d2befe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:26.322721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef582107aa244aa5a32ddbcf44d3444a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:26.323369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef582107aa244aa5a32ddbcf44d3444a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:26.324143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef582107aa244aa5a32ddbcf44d3444a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:26.339691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef582107aa244aa5a32ddbcf44d3444a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:26.363227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:26.363496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1063/2121] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:26.373718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1059/2122] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:25 2022] DELETE /identity/v3/users/ef582107aa244aa5a32ddbcf44d3444a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:26.378396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c097176-3d80-436a-ac12-e2895d2befe9 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:26.378396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2c097176-3d80-436a-ac12-e2895d2befe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:26.452036 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c097176-3d80-436a-ac12-e2895d2befe9 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:26.525897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af2d2f-6e6e-4216-9da1-c34a00c496b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:26.526562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af2d2f-6e6e-4216-9da1-c34a00c496b3 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:26.526834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af2d2f-6e6e-4216-9da1-c34a00c496b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:26.527107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af2d2f-6e6e-4216-9da1-c34a00c496b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:26.590114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af2d2f-6e6e-4216-9da1-c34a00c496b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:26.591806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af2d2f-6e6e-4216-9da1-c34a00c496b3 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:26.603391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1064/2123] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:26.633703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:26.999555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:27.001067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:27.080858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044efb9-1d6c-4bd8-9c0b-6b4dd47859cb admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:27.086106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:27.088549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:27.088549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:27.088549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] PATH_INFO: `/v3/projects/3cf1098cc6494c7aa9fa0e043d7b9f14` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:27.088549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf1098cc6494c7aa9fa0e043d7b9f14)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:27.090388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:27.209514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf1098cc6494c7aa9fa0e043d7b9f14', 'request_id': 'req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.216004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf1098cc6494c7aa9fa0e043d7b9f14', 'request_id': 'req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.325369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf1098cc6494c7aa9fa0e043d7b9f14', 'request_id': 'req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.339514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] The token cache is being invalidate because project 3cf1098cc6494c7aa9fa0e043d7b9f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:27.339813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.340514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-70eb4a36-7f30-4c13-b2e1-67fd7657fbc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.345271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1060/2124] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:26 2022] DELETE /identity/v3/projects/3cf1098cc6494c7aa9fa0e043d7b9f14 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:27.347239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af2d2f-6e6e-4216-9da1-c34a00c496b3 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:27.364355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af2d2f-6e6e-4216-9da1-c34a00c496b3 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:27.373842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813e71-ac03-4011-8918-742cc484f78d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:27.374882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813e71-ac03-4011-8918-742cc484f78d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:27.375169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813e71-ac03-4011-8918-742cc484f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:27.415163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0d813e71-ac03-4011-8918-742cc484f78d None None] MFA Rules not processed for user `793674c65b4147b39b94bacb791e3286`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:27.416674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0d813e71-ac03-4011-8918-742cc484f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:27.456467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:27.457796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:27.458404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:27.458692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] PATH_INFO: `/v3/users/2333f82197a843cd9b80fb2f9c89273f` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:27.459938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2333f82197a843cd9b80fb2f9c89273f)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:27.461941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:27.464023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:27.510052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1061/2125] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:27.522695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d813e71-ac03-4011-8918-742cc484f78d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:27.524107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0d813e71-ac03-4011-8918-742cc484f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:27.545919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:27.603202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d813e71-ac03-4011-8918-742cc484f78d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:27.608669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:27.609489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:27.609828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:27.610205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:27.612066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:27.676666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2333f82197a843cd9b80fb2f9c89273f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.681458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:27.683133 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:27.692786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1062/2126] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:27.705462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:27.705462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:27.721653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97078da-b869-4a91-8ff6-6b01f25cf388 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:27.727725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:27.728025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:27.728305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:27.728582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] PATH_INFO: `/v3/users/8026633a0f004a078a0daad083b78bfa` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:27.729938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8026633a0f004a078a0daad083b78bfa)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:27.738195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:27.800238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2333f82197a843cd9b80fb2f9c89273f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.800962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2333f82197a843cd9b80fb2f9c89273f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.814542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2333f82197a843cd9b80fb2f9c89273f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.821564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-225694b0-1ed0-4028-b4cd-f403bb75c73f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2333f82197a843cd9b80fb2f9c89273f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:27.828155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1065/2127] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:27 2022] DELETE /identity/v3/users/2333f82197a843cd9b80fb2f9c89273f => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:27.967395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8026633a0f004a078a0daad083b78bfa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.034028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8026633a0f004a078a0daad083b78bfa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.035365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8026633a0f004a078a0daad083b78bfa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.035986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8026633a0f004a078a0daad083b78bfa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.044742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8026633a0f004a078a0daad083b78bfa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.069112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1063/2128] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:27 2022] DELETE /identity/v3/users/8026633a0f004a078a0daad083b78bfa => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:28.073827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf895445-be1f-464a-bc1b-fb72825f1c34 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.074243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf895445-be1f-464a-bc1b-fb72825f1c34 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.074576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf895445-be1f-464a-bc1b-fb72825f1c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.096354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-bf895445-be1f-464a-bc1b-fb72825f1c34 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:28.097683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bf895445-be1f-464a-bc1b-fb72825f1c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.159476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1066/2129] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:28.169843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:28.171345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.229753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac61d-f1f8-43d5-9fa1-2b4762c53e2d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:28.233535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138618c-d5fb-4c8e-8739-de9d2a459eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:28.234166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138618c-d5fb-4c8e-8739-de9d2a459eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.234452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138618c-d5fb-4c8e-8739-de9d2a459eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.234732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138618c-d5fb-4c8e-8739-de9d2a459eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.244400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138618c-d5fb-4c8e-8739-de9d2a459eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:28.245833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138618c-d5fb-4c8e-8739-de9d2a459eb4 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:28.255942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1064/2130] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:28.292084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf895445-be1f-464a-bc1b-fb72825f1c34 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:28.293616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bf895445-be1f-464a-bc1b-fb72825f1c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.306635 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf895445-be1f-464a-bc1b-fb72825f1c34 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:28.312734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:28.313491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.313808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.314168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] PATH_INFO: `/v3/projects/d58c574ef2824ee497a75cf7d8276e5c` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.315466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58c574ef2824ee497a75cf7d8276e5c)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:28.318173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:28.437461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58c574ef2824ee497a75cf7d8276e5c', 'request_id': 'req-1661d7ce-bac6-47e7-be81-5a2796eaec42'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.447646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58c574ef2824ee497a75cf7d8276e5c', 'request_id': 'req-1661d7ce-bac6-47e7-be81-5a2796eaec42'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.556330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58c574ef2824ee497a75cf7d8276e5c', 'request_id': 'req-1661d7ce-bac6-47e7-be81-5a2796eaec42'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.568968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] The token cache is being invalidate because project d58c574ef2824ee497a75cf7d8276e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:28.569520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.570547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1661d7ce-bac6-47e7-be81-5a2796eaec42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.572277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1067/2131] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:28 2022] DELETE /identity/v3/projects/d58c574ef2824ee497a75cf7d8276e5c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:28.579975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ff34-dfec-4985-9b9c-56dd55fd0e17 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.579975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ff34-dfec-4985-9b9c-56dd55fd0e17 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.579975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ff34-dfec-4985-9b9c-56dd55fd0e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.603585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0852ff34-dfec-4985-9b9c-56dd55fd0e17 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:28.604975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0852ff34-dfec-4985-9b9c-56dd55fd0e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.667819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1065/2132] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:28.676593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd627477-d07a-428c-b623-0466ed18007d None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.676990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd627477-d07a-428c-b623-0466ed18007d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.677269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd627477-d07a-428c-b623-0466ed18007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.685051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:28.697514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-fd627477-d07a-428c-b623-0466ed18007d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:28.698738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fd627477-d07a-428c-b623-0466ed18007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.706732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1068/2133] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:28.710998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852ff34-dfec-4985-9b9c-56dd55fd0e17 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:28.723980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852ff34-dfec-4985-9b9c-56dd55fd0e17 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:28.726775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:28.727387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.727671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.727945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.728598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:28.728947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:28.730587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:28.743785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1066/2134] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:55:28.749389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:28.749586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd627477-d07a-428c-b623-0466ed18007d None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:28.751263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fd627477-d07a-428c-b623-0466ed18007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.760887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bf396-f3e7-42bf-95eb-6da3e363cc01 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:28.764137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd627477-d07a-428c-b623-0466ed18007d None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:28.766787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bf4dc-608d-49d0-ba66-0579c23ffa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:28.767605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bf4dc-608d-49d0-ba66-0579c23ffa85 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.767899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bf4dc-608d-49d0-ba66-0579c23ffa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.768445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bf4dc-608d-49d0-ba66-0579c23ffa85 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.769113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bf4dc-608d-49d0-ba66-0579c23ffa85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:28.770220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:28.775557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.775557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.775557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] PATH_INFO: `/v3/projects/d87aab0773ff4e4d9d90bd7d8f9943da` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.775852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bf4dc-608d-49d0-ba66-0579c23ffa85 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:28.775946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87aab0773ff4e4d9d90bd7d8f9943da)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:28.775946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:28.880967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1067/2135] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:28 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:28.882309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bf4dc-608d-49d0-ba66-0579c23ffa85 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:28.922374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87aab0773ff4e4d9d90bd7d8f9943da', 'request_id': 'req-61115a94-560c-44fe-9b72-3685f3b7ef78'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.930795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87aab0773ff4e4d9d90bd7d8f9943da', 'request_id': 'req-61115a94-560c-44fe-9b72-3685f3b7ef78'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:28.955239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bf4dc-608d-49d0-ba66-0579c23ffa85 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:29.010277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87aab0773ff4e4d9d90bd7d8f9943da', 'request_id': 'req-61115a94-560c-44fe-9b72-3685f3b7ef78'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:29.017860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:29.018809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:29.019083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:29.019360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:29.030533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:29.030533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:29.030742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] The token cache is being invalidate because project d87aab0773ff4e4d9d90bd7d8f9943da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:29.030742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:29.030742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:29.030742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1069/2136] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:28 2022] DELETE /identity/v3/projects/d87aab0773ff4e4d9d90bd7d8f9943da => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:29.032984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:29.033744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:29.047600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61115a94-560c-44fe-9b72-3685f3b7ef78 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:29.052869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:29.053506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:29.053785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:29.054057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] PATH_INFO: `/v3/users/5b977d2e68e44587af9b3d06f4323119` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:29.055332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b977d2e68e44587af9b3d06f4323119)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:29.057402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:29.168477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1068/2137] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:28 2022] POST /identity/v3/users => generated 295 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:29.170522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:29.172249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:29.184112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29046-b8fa-46d8-a715-a5e3610eed8a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:29.190090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdb1b3-1ede-4103-991c-dda258235045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:29.190786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdb1b3-1ede-4103-991c-dda258235045 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:29.191066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdb1b3-1ede-4103-991c-dda258235045 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:29.191342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdb1b3-1ede-4103-991c-dda258235045 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:29.192015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdb1b3-1ede-4103-991c-dda258235045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:29.193631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdb1b3-1ede-4103-991c-dda258235045 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:29.212513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1069/2138] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:29.213640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdb1b3-1ede-4103-991c-dda258235045 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:29.227395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdb1b3-1ede-4103-991c-dda258235045 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:29.233279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47082fb2-87af-48d2-b8d2-84aa35a5f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:29.234366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47082fb2-87af-48d2-b8d2-84aa35a5f19c admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:29.234366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47082fb2-87af-48d2-b8d2-84aa35a5f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:29.234563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47082fb2-87af-48d2-b8d2-84aa35a5f19c admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:29.235213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47082fb2-87af-48d2-b8d2-84aa35a5f19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:29.236889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47082fb2-87af-48d2-b8d2-84aa35a5f19c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:29.252526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1070/2139] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:29.253546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47082fb2-87af-48d2-b8d2-84aa35a5f19c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:29.263536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47082fb2-87af-48d2-b8d2-84aa35a5f19c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:29.268988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3de66c-407e-4af9-b898-ae787a944174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:29.270166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3de66c-407e-4af9-b898-ae787a944174 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:29.270458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3de66c-407e-4af9-b898-ae787a944174 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:29.270803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3de66c-407e-4af9-b898-ae787a944174 admin admin] PATH_INFO: `/v3/projects/e62a77dc2ac8432d93b94431f807ec15/users/985b73bbf878487abd05344d6a304c7e/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:29.288395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3de66c-407e-4af9-b898-ae787a944174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62a77dc2ac8432d93b94431f807ec15, user_id=985b73bbf878487abd05344d6a304c7e, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:29.291253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3de66c-407e-4af9-b898-ae787a944174 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:29.293460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b977d2e68e44587af9b3d06f4323119'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:29.301853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:29.397706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b977d2e68e44587af9b3d06f4323119'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:29.399230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b977d2e68e44587af9b3d06f4323119'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:29.409165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1071/2140] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:29 2022] PUT /identity/v3/projects/e62a77dc2ac8432d93b94431f807ec15/users/985b73bbf878487abd05344d6a304c7e/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:29.420302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b977d2e68e44587af9b3d06f4323119'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:29.429818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b977d2e68e44587af9b3d06f4323119'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:29.436550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1070/2141] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:29 2022] DELETE /identity/v3/users/5b977d2e68e44587af9b3d06f4323119 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:29.482221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba783-647e-46e5-8b61-d8b7ac8ee03f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:29.482512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba783-647e-46e5-8b61-d8b7ac8ee03f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:29.482789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba783-647e-46e5-8b61-d8b7ac8ee03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:29.515627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-68bba783-647e-46e5-8b61-d8b7ac8ee03f None None] MFA Rules not processed for user `985b73bbf878487abd05344d6a304c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:29.517008 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-68bba783-647e-46e5-8b61-d8b7ac8ee03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:29.573431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:29.575975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:29.604553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1072/2142] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:29.615366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bba783-647e-46e5-8b61-d8b7ac8ee03f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:29.616776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-68bba783-647e-46e5-8b61-d8b7ac8ee03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:29.650685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86af4d-a885-4b8d-9128-1ac3a2a67244 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:29.654967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e967cf-0c12-458a-b7e9-3a0eb2fbb38e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:29.659846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e967cf-0c12-458a-b7e9-3a0eb2fbb38e service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:29.659846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e967cf-0c12-458a-b7e9-3a0eb2fbb38e service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:29.659846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e967cf-0c12-458a-b7e9-3a0eb2fbb38e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:29.659846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c3e967cf-0c12-458a-b7e9-3a0eb2fbb38e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:29.686226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bba783-647e-46e5-8b61-d8b7ac8ee03f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:29.690791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:29.691950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:29.692230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:29.692504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:29.703092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:29.706065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:29.717955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1073/2143] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:29.752572 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e967cf-0c12-458a-b7e9-3a0eb2fbb38e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:29.754827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e967cf-0c12-458a-b7e9-3a0eb2fbb38e service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:29.771742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1071/2144] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:29.959377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:29.961919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:30.073975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f052ff1-48ae-46a5-b7db-4745b5a0c94d service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:30.083040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:30.083761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.084045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.084376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] PATH_INFO: `/v3/projects/617637240a5645b6b36e829c59e1ce55` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.086456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=617637240a5645b6b36e829c59e1ce55)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:30.089280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:30.226751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '617637240a5645b6b36e829c59e1ce55', 'request_id': 'req-1f933398-37db-4992-a375-c4921999e2c3'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:30.234161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '617637240a5645b6b36e829c59e1ce55', 'request_id': 'req-1f933398-37db-4992-a375-c4921999e2c3'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:30.341517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '617637240a5645b6b36e829c59e1ce55', 'request_id': 'req-1f933398-37db-4992-a375-c4921999e2c3'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:30.353157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] The token cache is being invalidate because project 617637240a5645b6b36e829c59e1ce55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:30.353448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:30.354089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-1f933398-37db-4992-a375-c4921999e2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:30.355886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1074/2145] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:29 2022] DELETE /identity/v3/projects/617637240a5645b6b36e829c59e1ce55 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:30.361024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114d76-80fa-4296-ad41-481694b1863e None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.366598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114d76-80fa-4296-ad41-481694b1863e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.366598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114d76-80fa-4296-ad41-481694b1863e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.383780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-3a114d76-80fa-4296-ad41-481694b1863e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:30.385010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3a114d76-80fa-4296-ad41-481694b1863e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:30.443755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:30.449250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1072/2146] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:30.458663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe93f-4d61-4c35-97d3-22911a6f6892 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.458956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe93f-4d61-4c35-97d3-22911a6f6892 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.459253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe93f-4d61-4c35-97d3-22911a6f6892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.479882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-254fe93f-4d61-4c35-97d3-22911a6f6892 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:30.481114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-254fe93f-4d61-4c35-97d3-22911a6f6892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:30.489941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1075/2147] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:30.494488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114d76-80fa-4296-ad41-481694b1863e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:30.504778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114d76-80fa-4296-ad41-481694b1863e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:30.509679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:30.510331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.510618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.510892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.511556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:30.511875 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:30.513494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:30.527430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1073/2148] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:55:30.532353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fe93f-4d61-4c35-97d3-22911a6f6892 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:30.543498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fe93f-4d61-4c35-97d3-22911a6f6892 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:30.549402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff788dc-c3c2-4c80-8ce2-8aab64461e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:30.550661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff788dc-c3c2-4c80-8ce2-8aab64461e9f admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.551166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff788dc-c3c2-4c80-8ce2-8aab64461e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.551384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff788dc-c3c2-4c80-8ce2-8aab64461e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.552267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff788dc-c3c2-4c80-8ce2-8aab64461e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:30.554680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff788dc-c3c2-4c80-8ce2-8aab64461e9f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:30.638221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1076/2149] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:30 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:30.640240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:30.651067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8244a-c1ce-4ec6-b393-e4c7e67edf3c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:30.709721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1020b-6fd8-443d-82d2-cfb8dcf90d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:30.710381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1020b-6fd8-443d-82d2-cfb8dcf90d87 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.710658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1020b-6fd8-443d-82d2-cfb8dcf90d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.710933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1020b-6fd8-443d-82d2-cfb8dcf90d87 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.711575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1020b-6fd8-443d-82d2-cfb8dcf90d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:30.713504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1020b-6fd8-443d-82d2-cfb8dcf90d87 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:30.803309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:30.812033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1074/2150] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:30 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:30.815464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff788dc-c3c2-4c80-8ce2-8aab64461e9f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:30.828770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff788dc-c3c2-4c80-8ce2-8aab64461e9f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:30.850623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1caea-f2ff-41ed-93f7-52674b12043d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:30.851343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1caea-f2ff-41ed-93f7-52674b12043d admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.851690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1caea-f2ff-41ed-93f7-52674b12043d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.852028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1caea-f2ff-41ed-93f7-52674b12043d admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.852833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1caea-f2ff-41ed-93f7-52674b12043d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:30.854823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1caea-f2ff-41ed-93f7-52674b12043d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:30.868046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1077/2151] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:30.872014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1020b-6fd8-443d-82d2-cfb8dcf90d87 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:30.884471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1020b-6fd8-443d-82d2-cfb8dcf90d87 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:30.890680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05652d09-01a8-4080-8eb7-a80b08d49db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:30.894178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05652d09-01a8-4080-8eb7-a80b08d49db5 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.894178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05652d09-01a8-4080-8eb7-a80b08d49db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.894178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05652d09-01a8-4080-8eb7-a80b08d49db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.894178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05652d09-01a8-4080-8eb7-a80b08d49db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:30.895291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05652d09-01a8-4080-8eb7-a80b08d49db5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:30.906779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1075/2152] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:30.913778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1caea-f2ff-41ed-93f7-52674b12043d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:30.924051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1caea-f2ff-41ed-93f7-52674b12043d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:30.929269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43d8d8-8847-42bd-b980-590ca76c5cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:30.929938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d8d8-8847-42bd-b980-590ca76c5cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:30.930253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d8d8-8847-42bd-b980-590ca76c5cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:30.930532 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d8d8-8847-42bd-b980-590ca76c5cc3 admin admin] PATH_INFO: `/v3/projects/c68ca73a8e874c23bc32977c25e1d494/users/3476226c1d7d4da68408281d7d75ca19/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:30.948277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43d8d8-8847-42bd-b980-590ca76c5cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68ca73a8e874c23bc32977c25e1d494, user_id=3476226c1d7d4da68408281d7d75ca19, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:30.950630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43d8d8-8847-42bd-b980-590ca76c5cc3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:31.025860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1078/2153] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:30 2022] PUT /identity/v3/projects/c68ca73a8e874c23bc32977c25e1d494/users/3476226c1d7d4da68408281d7d75ca19/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:31.037846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3be0b-f25f-4ba2-a437-be5beb441cc0 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.038180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3be0b-f25f-4ba2-a437-be5beb441cc0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.038465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3be0b-f25f-4ba2-a437-be5beb441cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.067834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-57e3be0b-f25f-4ba2-a437-be5beb441cc0 None None] MFA Rules not processed for user `3476226c1d7d4da68408281d7d75ca19`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:31.069067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-57e3be0b-f25f-4ba2-a437-be5beb441cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:31.136725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1076/2154] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:31.141318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43d8d8-8847-42bd-b980-590ca76c5cc3 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:31.206439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43d8d8-8847-42bd-b980-590ca76c5cc3 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:31.212017 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29920db8-4033-4ad8-b202-a883c95c5887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:31.212659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920db8-4033-4ad8-b202-a883c95c5887 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.213076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920db8-4033-4ad8-b202-a883c95c5887 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.213362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920db8-4033-4ad8-b202-a883c95c5887 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.214025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29920db8-4033-4ad8-b202-a883c95c5887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:31.216014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29920db8-4033-4ad8-b202-a883c95c5887 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:31.354061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1079/2155] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:31 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:31.358985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3be0b-f25f-4ba2-a437-be5beb441cc0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:31.372470 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3be0b-f25f-4ba2-a437-be5beb441cc0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:31.440593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b83b2-ab51-434a-b880-b87c5df8822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:31.441191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b83b2-ab51-434a-b880-b87c5df8822f admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.441521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b83b2-ab51-434a-b880-b87c5df8822f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.442352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b83b2-ab51-434a-b880-b87c5df8822f admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.442990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b83b2-ab51-434a-b880-b87c5df8822f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:31.444915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b83b2-ab51-434a-b880-b87c5df8822f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:31.553155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1077/2156] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:31 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:31.557564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29920db8-4033-4ad8-b202-a883c95c5887 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:31.569141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29920db8-4033-4ad8-b202-a883c95c5887 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:31.574941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224904e-1b61-422a-95f6-8a527969f2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:31.577363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224904e-1b61-422a-95f6-8a527969f2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.577363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224904e-1b61-422a-95f6-8a527969f2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.577363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224904e-1b61-422a-95f6-8a527969f2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.577363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224904e-1b61-422a-95f6-8a527969f2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:31.579064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224904e-1b61-422a-95f6-8a527969f2d9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:31.592273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1080/2157] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:31.599377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b83b2-ab51-434a-b880-b87c5df8822f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:31.611347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b83b2-ab51-434a-b880-b87c5df8822f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:31.617880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e932c3-c022-4a2f-8293-cdbfde1c9a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:31.619027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e932c3-c022-4a2f-8293-cdbfde1c9a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.623411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e932c3-c022-4a2f-8293-cdbfde1c9a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.623411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e932c3-c022-4a2f-8293-cdbfde1c9a02 admin admin] PATH_INFO: `/v3/projects/269530edc7c748ce8e18457c32cbd03c/users/1fc6f3ff3f2549939c08d82fe85e2d6c/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.635695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e932c3-c022-4a2f-8293-cdbfde1c9a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=269530edc7c748ce8e18457c32cbd03c, user_id=1fc6f3ff3f2549939c08d82fe85e2d6c, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:31.638607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e932c3-c022-4a2f-8293-cdbfde1c9a02 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:31.753973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1078/2158] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:31 2022] PUT /identity/v3/projects/269530edc7c748ce8e18457c32cbd03c/users/1fc6f3ff3f2549939c08d82fe85e2d6c/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:31.758407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224904e-1b61-422a-95f6-8a527969f2d9 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:31.768760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224904e-1b61-422a-95f6-8a527969f2d9 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:31.809632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:31.853124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783821d-29ea-439f-a2e7-0fd284e90bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:31.853983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783821d-29ea-439f-a2e7-0fd284e90bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.854395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783821d-29ea-439f-a2e7-0fd284e90bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.854787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783821d-29ea-439f-a2e7-0fd284e90bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.855759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783821d-29ea-439f-a2e7-0fd284e90bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:31.859906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783821d-29ea-439f-a2e7-0fd284e90bd8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:31.873892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1081/2159] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:31.879691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e932c3-c022-4a2f-8293-cdbfde1c9a02 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:31.894775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e932c3-c022-4a2f-8293-cdbfde1c9a02 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:31.901263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:31.902200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.902578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.903013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] PATH_INFO: `/v3/projects/269530edc7c748ce8e18457c32cbd03c/users/1fc6f3ff3f2549939c08d82fe85e2d6c/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.905082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=269530edc7c748ce8e18457c32cbd03c, user_id=1fc6f3ff3f2549939c08d82fe85e2d6c, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:31.908217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:31.924648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1079/2160] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:31 2022] PUT /identity/v3/projects/269530edc7c748ce8e18457c32cbd03c/users/1fc6f3ff3f2549939c08d82fe85e2d6c/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:31.939664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e46e-6fb5-4c4c-84dc-26a577bf1354 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.940893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e46e-6fb5-4c4c-84dc-26a577bf1354 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.941257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e46e-6fb5-4c4c-84dc-26a577bf1354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.982795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-4535e46e-6fb5-4c4c-84dc-26a577bf1354 None None] MFA Rules not processed for user `1fc6f3ff3f2549939c08d82fe85e2d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:31.984086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4535e46e-6fb5-4c4c-84dc-26a577bf1354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:32.074226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1082/2161] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:32.086246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:32.088042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:32.163239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0bdb5-59a3-4924-9ac4-9470141758b9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:32.167677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e0b95-1ef5-4703-9539-00481348e18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:32.168412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e0b95-1ef5-4703-9539-00481348e18a service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:32.168888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e0b95-1ef5-4703-9539-00481348e18a service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:32.169373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e0b95-1ef5-4703-9539-00481348e18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:32.180438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e0b95-1ef5-4703-9539-00481348e18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:32.181882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e0b95-1ef5-4703-9539-00481348e18a service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:32.192442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1080/2162] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:32.230078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535e46e-6fb5-4c4c-84dc-26a577bf1354 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:32.257011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:32.306053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535e46e-6fb5-4c4c-84dc-26a577bf1354 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:32.311641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:32.312333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:32.312616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:32.312890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] PATH_INFO: `/v3/users/3476226c1d7d4da68408281d7d75ca19` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:32.314080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3476226c1d7d4da68408281d7d75ca19)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:32.316076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:32.555543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3476226c1d7d4da68408281d7d75ca19'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:32.668277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3476226c1d7d4da68408281d7d75ca19'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:32.669208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3476226c1d7d4da68408281d7d75ca19'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:32.670052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3476226c1d7d4da68408281d7d75ca19'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:32.679558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3476226c1d7d4da68408281d7d75ca19'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:32.697380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1083/2163] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:32 2022] DELETE /identity/v3/users/3476226c1d7d4da68408281d7d75ca19 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:32.702066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfb3e0-6fa3-4921-a0e4-bfbe70fbd2e6 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:32.705824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfb3e0-6fa3-4921-a0e4-bfbe70fbd2e6 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:32.705824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfb3e0-6fa3-4921-a0e4-bfbe70fbd2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:32.722858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-42bfb3e0-6fa3-4921-a0e4-bfbe70fbd2e6 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:32.725020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-42bfb3e0-6fa3-4921-a0e4-bfbe70fbd2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:32.753268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:32.786125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1081/2164] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:32.795843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:32.797255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:32.861295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1329a-749c-4386-b1e0-106016fab86b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:32.865564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad17e2-dbbe-459f-84e8-6edc5b10efa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:32.866263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad17e2-dbbe-459f-84e8-6edc5b10efa1 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:32.866551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad17e2-dbbe-459f-84e8-6edc5b10efa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:32.866832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad17e2-dbbe-459f-84e8-6edc5b10efa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:32.881746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad17e2-dbbe-459f-84e8-6edc5b10efa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:32.883251 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad17e2-dbbe-459f-84e8-6edc5b10efa1 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:32.894011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1084/2165] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:32.912599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:33.377479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfb3e0-6fa3-4921-a0e4-bfbe70fbd2e6 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:33.379034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-42bfb3e0-6fa3-4921-a0e4-bfbe70fbd2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:33.390943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfb3e0-6fa3-4921-a0e4-bfbe70fbd2e6 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:33.397868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:33.398545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:33.403399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:33.403399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] PATH_INFO: `/v3/projects/c68ca73a8e874c23bc32977c25e1d494` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:33.403399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68ca73a8e874c23bc32977c25e1d494)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:33.403399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:33.540518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68ca73a8e874c23bc32977c25e1d494', 'request_id': 'req-e291e353-e87b-45b8-b526-2705479b4d5b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.547145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68ca73a8e874c23bc32977c25e1d494', 'request_id': 'req-e291e353-e87b-45b8-b526-2705479b4d5b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.632866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68ca73a8e874c23bc32977c25e1d494', 'request_id': 'req-e291e353-e87b-45b8-b526-2705479b4d5b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.647871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] The token cache is being invalidate because project c68ca73a8e874c23bc32977c25e1d494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:33.648245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.648977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.651166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1082/2166] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:33 2022] DELETE /identity/v3/projects/c68ca73a8e874c23bc32977c25e1d494 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:33.655090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad17e2-dbbe-459f-84e8-6edc5b10efa1 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:33.666570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad17e2-dbbe-459f-84e8-6edc5b10efa1 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:33.728639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:33.729200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:33.729478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:33.729752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] PATH_INFO: `/v3/users/1fc6f3ff3f2549939c08d82fe85e2d6c` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:33.730924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc6f3ff3f2549939c08d82fe85e2d6c)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:33.732997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:33.827717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:33.899943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc6f3ff3f2549939c08d82fe85e2d6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.987519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc6f3ff3f2549939c08d82fe85e2d6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.988086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc6f3ff3f2549939c08d82fe85e2d6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.988704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc6f3ff3f2549939c08d82fe85e2d6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:33.998554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-47633be6-3f43-4846-a9d8-03bcf1004d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc6f3ff3f2549939c08d82fe85e2d6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:34.008566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:34.027139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1085/2167] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:33 2022] DELETE /identity/v3/users/1fc6f3ff3f2549939c08d82fe85e2d6c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:34.435535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:34.437016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:34.452458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e353-e87b-45b8-b526-2705479b4d5b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:34.513668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:34.514244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:34.514524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:34.514798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] PATH_INFO: `/v3/projects/269530edc7c748ce8e18457c32cbd03c` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:34.521168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=269530edc7c748ce8e18457c32cbd03c)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:34.523677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:34.540286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:34.672518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '269530edc7c748ce8e18457c32cbd03c', 'request_id': 'req-0a10438d-afc0-4a7c-8505-8aec53201df8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:34.679372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '269530edc7c748ce8e18457c32cbd03c', 'request_id': 'req-0a10438d-afc0-4a7c-8505-8aec53201df8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:34.759284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '269530edc7c748ce8e18457c32cbd03c', 'request_id': 'req-0a10438d-afc0-4a7c-8505-8aec53201df8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:34.773831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] The token cache is being invalidate because project 269530edc7c748ce8e18457c32cbd03c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:34.773831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:34.773831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0a10438d-afc0-4a7c-8505-8aec53201df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:34.773831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1083/2168] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:34 2022] DELETE /identity/v3/projects/269530edc7c748ce8e18457c32cbd03c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:34.778274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101245-9183-4e2a-a5e6-82a373090312 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:34.778590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101245-9183-4e2a-a5e6-82a373090312 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:34.778872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101245-9183-4e2a-a5e6-82a373090312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:34.799120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-d9101245-9183-4e2a-a5e6-82a373090312 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:34.800338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d9101245-9183-4e2a-a5e6-82a373090312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:34.869386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1086/2169] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:34.871128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6f35-ec9d-4725-8287-6766d8c19a08 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:34.871418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6f35-ec9d-4725-8287-6766d8c19a08 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:34.871690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6f35-ec9d-4725-8287-6766d8c19a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:34.892507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c6da6f35-ec9d-4725-8287-6766d8c19a08 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:34.894247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c6da6f35-ec9d-4725-8287-6766d8c19a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:34.906368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1084/2170] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:34.908021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9101245-9183-4e2a-a5e6-82a373090312 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:34.922594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9101245-9183-4e2a-a5e6-82a373090312 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:34.928798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:34.929448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:34.929732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:34.930002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:34.930939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:34.931244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:34.932846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:34.950245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1087/2171] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:55:34.960010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da6f35-ec9d-4725-8287-6766d8c19a08 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:34.982650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da6f35-ec9d-4725-8287-6766d8c19a08 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:34.988207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcee840-9c2b-4e38-b2a8-b07868c99ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:34.989461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcee840-9c2b-4e38-b2a8-b07868c99ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:34.989670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcee840-9c2b-4e38-b2a8-b07868c99ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:34.989939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcee840-9c2b-4e38-b2a8-b07868c99ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:34.990610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcee840-9c2b-4e38-b2a8-b07868c99ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:34.993043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcee840-9c2b-4e38-b2a8-b07868c99ee0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.117112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1085/2172] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:34 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:35.118453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.128375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3935c3e-3b67-4feb-b66e-8f63609335a2 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:35.190216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8886-3be6-4df0-a774-4f1008b1a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:35.190866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8886-3be6-4df0-a774-4f1008b1a6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.191186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8886-3be6-4df0-a774-4f1008b1a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.191467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8886-3be6-4df0-a774-4f1008b1a6d0 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.192428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8886-3be6-4df0-a774-4f1008b1a6d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:35.194097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8886-3be6-4df0-a774-4f1008b1a6d0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.291760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1088/2173] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:35 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:35.296174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcee840-9c2b-4e38-b2a8-b07868c99ee0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.306587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcee840-9c2b-4e38-b2a8-b07868c99ee0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:35.311643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ab7fe-20d4-4204-8884-1d9c97a7db99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:35.311894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab7fe-20d4-4204-8884-1d9c97a7db99 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.312088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab7fe-20d4-4204-8884-1d9c97a7db99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.312363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab7fe-20d4-4204-8884-1d9c97a7db99 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.313010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ab7fe-20d4-4204-8884-1d9c97a7db99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:35.314671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ab7fe-20d4-4204-8884-1d9c97a7db99 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.324400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1086/2174] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:35.328363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8886-3be6-4df0-a774-4f1008b1a6d0 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.339211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8886-3be6-4df0-a774-4f1008b1a6d0 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:35.344086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4effd-cdf3-488a-ab3b-38e7a143bfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:35.344684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4effd-cdf3-488a-ab3b-38e7a143bfee admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.344968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4effd-cdf3-488a-ab3b-38e7a143bfee admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.345262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4effd-cdf3-488a-ab3b-38e7a143bfee admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.345888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4effd-cdf3-488a-ab3b-38e7a143bfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:35.347517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4effd-cdf3-488a-ab3b-38e7a143bfee admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.358696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1089/2175] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:35.362939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ab7fe-20d4-4204-8884-1d9c97a7db99 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.372900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ab7fe-20d4-4204-8884-1d9c97a7db99 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:35.377784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f80d69-0b5e-49cf-8ebf-c5c274d535ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:35.378460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f80d69-0b5e-49cf-8ebf-c5c274d535ed admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.378756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f80d69-0b5e-49cf-8ebf-c5c274d535ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.379507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f80d69-0b5e-49cf-8ebf-c5c274d535ed admin admin] PATH_INFO: `/v3/projects/1752346a8fc94ef484548294dc9704ef/users/410ea99c613b46e28aee277eebcf56cc/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.394450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f80d69-0b5e-49cf-8ebf-c5c274d535ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1752346a8fc94ef484548294dc9704ef, user_id=410ea99c613b46e28aee277eebcf56cc, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:35.397766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f80d69-0b5e-49cf-8ebf-c5c274d535ed admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.467779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1087/2176] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:35 2022] PUT /identity/v3/projects/1752346a8fc94ef484548294dc9704ef/users/410ea99c613b46e28aee277eebcf56cc/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:35.479056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f43f5-9f10-45a5-b8cd-c5c6571d58f4 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.482358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f43f5-9f10-45a5-b8cd-c5c6571d58f4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.482358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f43f5-9f10-45a5-b8cd-c5c6571d58f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.487679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:35.512833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-4d0f43f5-9f10-45a5-b8cd-c5c6571d58f4 None None] MFA Rules not processed for user `410ea99c613b46e28aee277eebcf56cc`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:35.514068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4d0f43f5-9f10-45a5-b8cd-c5c6571d58f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:35.580547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1090/2177] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:35.586321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f80d69-0b5e-49cf-8ebf-c5c274d535ed admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.660971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f80d69-0b5e-49cf-8ebf-c5c274d535ed admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:35.666027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadbe1f-72af-4cae-9c94-7cf7ed1c916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:35.666653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadbe1f-72af-4cae-9c94-7cf7ed1c916a admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.666931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadbe1f-72af-4cae-9c94-7cf7ed1c916a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.667200 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadbe1f-72af-4cae-9c94-7cf7ed1c916a admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.667858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadbe1f-72af-4cae-9c94-7cf7ed1c916a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:35.669780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadbe1f-72af-4cae-9c94-7cf7ed1c916a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.774697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1088/2178] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:35 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:35.778908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f43f5-9f10-45a5-b8cd-c5c6571d58f4 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.790687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f43f5-9f10-45a5-b8cd-c5c6571d58f4 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:35.850671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b24c-bb1b-4f73-bca4-b470ebd2c7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:35.851282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b24c-bb1b-4f73-bca4-b470ebd2c7ee admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.851561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b24c-bb1b-4f73-bca4-b470ebd2c7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.851897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b24c-bb1b-4f73-bca4-b470ebd2c7ee admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.852679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98b24c-bb1b-4f73-bca4-b470ebd2c7ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:35.854616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98b24c-bb1b-4f73-bca4-b470ebd2c7ee admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.946636 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1091/2179] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:35 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:35.950741 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadbe1f-72af-4cae-9c94-7cf7ed1c916a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.961401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadbe1f-72af-4cae-9c94-7cf7ed1c916a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:35.966409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be5c1-51ec-4312-81f0-41904d134d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:35.967056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be5c1-51ec-4312-81f0-41904d134d55 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.970225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be5c1-51ec-4312-81f0-41904d134d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.970225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be5c1-51ec-4312-81f0-41904d134d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.970225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618be5c1-51ec-4312-81f0-41904d134d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:35.970225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618be5c1-51ec-4312-81f0-41904d134d55 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.981789 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1089/2180] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:35.986029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b24c-bb1b-4f73-bca4-b470ebd2c7ee admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.997090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b24c-bb1b-4f73-bca4-b470ebd2c7ee admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:36.001811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c60a0-49b4-41f0-aacc-f9a3f1d4f9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:36.002472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c60a0-49b4-41f0-aacc-f9a3f1d4f9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.002756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c60a0-49b4-41f0-aacc-f9a3f1d4f9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.003033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c60a0-49b4-41f0-aacc-f9a3f1d4f9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.003681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c60a0-49b4-41f0-aacc-f9a3f1d4f9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:36.005934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c60a0-49b4-41f0-aacc-f9a3f1d4f9b7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:36.016280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1092/2181] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:36.021020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be5c1-51ec-4312-81f0-41904d134d55 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:36.030956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be5c1-51ec-4312-81f0-41904d134d55 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:36.035569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db6b0a-b20d-443f-a547-3b397b515023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:36.036223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db6b0a-b20d-443f-a547-3b397b515023 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.036504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db6b0a-b20d-443f-a547-3b397b515023 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.037194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db6b0a-b20d-443f-a547-3b397b515023 admin admin] PATH_INFO: `/v3/projects/179adadc4935466d836ae89ab11fd634/users/e8ae2af8ea684ca9a9deb62ea3cebb58/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.050338 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:36.054258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db6b0a-b20d-443f-a547-3b397b515023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=179adadc4935466d836ae89ab11fd634, user_id=e8ae2af8ea684ca9a9deb62ea3cebb58, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:36.056994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db6b0a-b20d-443f-a547-3b397b515023 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:36.178019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1090/2182] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:36 2022] PUT /identity/v3/projects/179adadc4935466d836ae89ab11fd634/users/e8ae2af8ea684ca9a9deb62ea3cebb58/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:36.188639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da67c-ca0c-4270-a10d-cfaee5d2c5cc None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.188875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da67c-ca0c-4270-a10d-cfaee5d2c5cc None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.189241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da67c-ca0c-4270-a10d-cfaee5d2c5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.233850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5c9da67c-ca0c-4270-a10d-cfaee5d2c5cc None None] MFA Rules not processed for user `e8ae2af8ea684ca9a9deb62ea3cebb58`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:36.235275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5c9da67c-ca0c-4270-a10d-cfaee5d2c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:36.298954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:36.303473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1093/2183] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:36.315302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db6b0a-b20d-443f-a547-3b397b515023 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:36.380589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db6b0a-b20d-443f-a547-3b397b515023 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:36.385263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917a7f-52e3-46a2-95a8-408553cdd1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:36.385870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917a7f-52e3-46a2-95a8-408553cdd1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.386164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917a7f-52e3-46a2-95a8-408553cdd1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.386468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917a7f-52e3-46a2-95a8-408553cdd1a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.387115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db917a7f-52e3-46a2-95a8-408553cdd1a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:36.389058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db917a7f-52e3-46a2-95a8-408553cdd1a6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:36.491713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1091/2184] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:36 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:36.495985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9da67c-ca0c-4270-a10d-cfaee5d2c5cc None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:36.507290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9da67c-ca0c-4270-a10d-cfaee5d2c5cc None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:36.579344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadf8a0-3d13-4c28-ab98-68afbe22f987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:36.579963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadf8a0-3d13-4c28-ab98-68afbe22f987 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.580239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadf8a0-3d13-4c28-ab98-68afbe22f987 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.580509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadf8a0-3d13-4c28-ab98-68afbe22f987 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.581394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadf8a0-3d13-4c28-ab98-68afbe22f987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:36.583962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadf8a0-3d13-4c28-ab98-68afbe22f987 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:36.709587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1094/2185] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:36 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:36.713153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917a7f-52e3-46a2-95a8-408553cdd1a6 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:36.727560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917a7f-52e3-46a2-95a8-408553cdd1a6 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:36.732208 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90b548-aac6-4722-9893-7bff0ad6ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:36.732820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90b548-aac6-4722-9893-7bff0ad6ea59 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.733101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90b548-aac6-4722-9893-7bff0ad6ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.733380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90b548-aac6-4722-9893-7bff0ad6ea59 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.734036 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90b548-aac6-4722-9893-7bff0ad6ea59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:36.735688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90b548-aac6-4722-9893-7bff0ad6ea59 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:36.747606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1092/2186] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:36.751897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadf8a0-3d13-4c28-ab98-68afbe22f987 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:36.761919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadf8a0-3d13-4c28-ab98-68afbe22f987 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:36.766684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35186f2e-eee6-4b60-8962-243c1e3829cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:36.767387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35186f2e-eee6-4b60-8962-243c1e3829cc admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.767665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35186f2e-eee6-4b60-8962-243c1e3829cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.767936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35186f2e-eee6-4b60-8962-243c1e3829cc admin admin] PATH_INFO: `/v3/projects/08eb1503034742339939ee950e7d96ef/users/0efd382e03664e58acdf004b120d506c/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.784184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35186f2e-eee6-4b60-8962-243c1e3829cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=08eb1503034742339939ee950e7d96ef, user_id=0efd382e03664e58acdf004b120d506c, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:36.787949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35186f2e-eee6-4b60-8962-243c1e3829cc admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:36.883415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1095/2187] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:36 2022] PUT /identity/v3/projects/08eb1503034742339939ee950e7d96ef/users/0efd382e03664e58acdf004b120d506c/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:36.888078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90b548-aac6-4722-9893-7bff0ad6ea59 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:36.899380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90b548-aac6-4722-9893-7bff0ad6ea59 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:36.956148 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5be34-b09d-412d-bf96-1f6bb2a1c640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:36.956680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5be34-b09d-412d-bf96-1f6bb2a1c640 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.956962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5be34-b09d-412d-bf96-1f6bb2a1c640 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.957869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5be34-b09d-412d-bf96-1f6bb2a1c640 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.958550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5be34-b09d-412d-bf96-1f6bb2a1c640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:36.960174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5be34-b09d-412d-bf96-1f6bb2a1c640 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:36.969920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1093/2188] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:36.974198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35186f2e-eee6-4b60-8962-243c1e3829cc admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:36.984713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35186f2e-eee6-4b60-8962-243c1e3829cc admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:36.989692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:36.990378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:36.990662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:36.990938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] PATH_INFO: `/v3/projects/08eb1503034742339939ee950e7d96ef/users/0efd382e03664e58acdf004b120d506c/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:36.992995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08eb1503034742339939ee950e7d96ef, user_id=0efd382e03664e58acdf004b120d506c, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:36.995701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:37.007150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1096/2189] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:36 2022] PUT /identity/v3/projects/08eb1503034742339939ee950e7d96ef/users/0efd382e03664e58acdf004b120d506c/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:37.017252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28717a-07af-4aa7-bcb4-779f3d5e6be9 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:37.017540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28717a-07af-4aa7-bcb4-779f3d5e6be9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:37.017818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28717a-07af-4aa7-bcb4-779f3d5e6be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:37.047446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ce28717a-07af-4aa7-bcb4-779f3d5e6be9 None None] MFA Rules not processed for user `0efd382e03664e58acdf004b120d506c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:37.048666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ce28717a-07af-4aa7-bcb4-779f3d5e6be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:37.101832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:37.122244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1094/2190] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:37.133540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:37.135979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:37.197361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ac31-a3cf-47cd-b681-da3106da9962 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:37.201145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:37.201761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:37.202045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:37.202956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:37.213457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:37.214417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:37.224490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1097/2191] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:38.373723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28717a-07af-4aa7-bcb4-779f3d5e6be9 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:38.383850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28717a-07af-4aa7-bcb4-779f3d5e6be9 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:38.387598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda6bc-1e28-49a2-a8d1-74535ec521c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:38.388201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda6bc-1e28-49a2-a8d1-74535ec521c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:38.388482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda6bc-1e28-49a2-a8d1-74535ec521c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:38.388752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda6bc-1e28-49a2-a8d1-74535ec521c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:38.447333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda6bc-1e28-49a2-a8d1-74535ec521c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:38.448769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda6bc-1e28-49a2-a8d1-74535ec521c0 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:38.458764 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1095/2192] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:47.220006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:47.768009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:47.769727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:47.842666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b9d19-15af-4f52-ad97-9475a29de11b service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:47.847518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:47.848196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:47.848479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:47.848760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] PATH_INFO: `/v3/users/62a404305ab046bea4d87a4d75bf203c` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:47.849901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a404305ab046bea4d87a4d75bf203c)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:47.851860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:48.069894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a404305ab046bea4d87a4d75bf203c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:48.145940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a404305ab046bea4d87a4d75bf203c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:48.146818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a404305ab046bea4d87a4d75bf203c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:48.147563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a404305ab046bea4d87a4d75bf203c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:48.156925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a404305ab046bea4d87a4d75bf203c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:48.182740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1098/2193] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:47 2022] DELETE /identity/v3/users/62a404305ab046bea4d87a4d75bf203c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:48.188903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f7541-fc25-4ca4-91cd-e450ef0d6dd2 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:48.189464 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f7541-fc25-4ca4-91cd-e450ef0d6dd2 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:48.189803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f7541-fc25-4ca4-91cd-e450ef0d6dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:48.211493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-680f7541-fc25-4ca4-91cd-e450ef0d6dd2 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:48.212981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-680f7541-fc25-4ca4-91cd-e450ef0d6dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:48.293138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1096/2194] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:48.306555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:48.308890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:48.372802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b87778-68c9-4748-826f-58653d448238 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:48.376461 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c055c6-0550-4b50-94c1-61e0a3f22644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:48.377066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c055c6-0550-4b50-94c1-61e0a3f22644 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:48.377369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c055c6-0550-4b50-94c1-61e0a3f22644 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:48.377626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c055c6-0550-4b50-94c1-61e0a3f22644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:48.386863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c055c6-0550-4b50-94c1-61e0a3f22644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:48.388308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c055c6-0550-4b50-94c1-61e0a3f22644 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:48.398917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1099/2195] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:48.908053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f7541-fc25-4ca4-91cd-e450ef0d6dd2 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:48.909543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-680f7541-fc25-4ca4-91cd-e450ef0d6dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:48.921622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f7541-fc25-4ca4-91cd-e450ef0d6dd2 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:48.942503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:48.943175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:48.943469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:48.943746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] PATH_INFO: `/v3/projects/e220224633104bcb865ddfab9e7fe033` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:48.947530 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e220224633104bcb865ddfab9e7fe033)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:48.949562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:49.074559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e220224633104bcb865ddfab9e7fe033', 'request_id': 'req-614f654e-e22b-48fd-b5de-514b502b5526'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.081218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e220224633104bcb865ddfab9e7fe033', 'request_id': 'req-614f654e-e22b-48fd-b5de-514b502b5526'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.143220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e220224633104bcb865ddfab9e7fe033', 'request_id': 'req-614f654e-e22b-48fd-b5de-514b502b5526'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.157320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] The token cache is being invalidate because project e220224633104bcb865ddfab9e7fe033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:49.157320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.157320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.158206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1097/2196] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:48 2022] DELETE /identity/v3/projects/e220224633104bcb865ddfab9e7fe033 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:49.162393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c055c6-0550-4b50-94c1-61e0a3f22644 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:49.172619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c055c6-0550-4b50-94c1-61e0a3f22644 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:49.194632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:49.238207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:49.239375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:49.239667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:49.239939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] PATH_INFO: `/v3/users/514e8cc06e094d6b9e15eaa010194caa` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:49.241123 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=514e8cc06e094d6b9e15eaa010194caa)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:49.243169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:49.463554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '514e8cc06e094d6b9e15eaa010194caa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.546459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '514e8cc06e094d6b9e15eaa010194caa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.547254 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '514e8cc06e094d6b9e15eaa010194caa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.548096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '514e8cc06e094d6b9e15eaa010194caa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.556943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-2ade9f53-1cd4-4d74-86da-66f0056d7533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '514e8cc06e094d6b9e15eaa010194caa'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:49.575733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1100/2197] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:49 2022] DELETE /identity/v3/users/514e8cc06e094d6b9e15eaa010194caa => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:50.061221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:50.062905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:50.078433 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f654e-e22b-48fd-b5de-514b502b5526 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:50.084335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:50.155035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:50.155713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:50.156007 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:50.156388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] PATH_INFO: `/v3/projects/05f822df365b4acbb0ba69cc4696bf80` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:50.157807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f822df365b4acbb0ba69cc4696bf80)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:50.160145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:50.308995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f822df365b4acbb0ba69cc4696bf80', 'request_id': 'req-72fb6da5-d53e-43bc-b3b8-48be4951f33a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:50.315422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f822df365b4acbb0ba69cc4696bf80', 'request_id': 'req-72fb6da5-d53e-43bc-b3b8-48be4951f33a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:50.374892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f822df365b4acbb0ba69cc4696bf80', 'request_id': 'req-72fb6da5-d53e-43bc-b3b8-48be4951f33a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:50.387075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] The token cache is being invalidate because project 05f822df365b4acbb0ba69cc4696bf80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:50.387370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:50.391305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-72fb6da5-d53e-43bc-b3b8-48be4951f33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:50.391305 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1098/2198] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:50 2022] DELETE /identity/v3/projects/05f822df365b4acbb0ba69cc4696bf80 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:50.394580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ee2d4-51fc-4717-8286-f5accd13cd54 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:50.395675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ee2d4-51fc-4717-8286-f5accd13cd54 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:50.395958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ee2d4-51fc-4717-8286-f5accd13cd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:50.417145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-cb4ee2d4-51fc-4717-8286-f5accd13cd54 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:50.418417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cb4ee2d4-51fc-4717-8286-f5accd13cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:50.486020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1101/2199] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:50.501302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18a039-71a2-4ecf-8028-e8bd93bd7117 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:50.501588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18a039-71a2-4ecf-8028-e8bd93bd7117 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:50.501866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18a039-71a2-4ecf-8028-e8bd93bd7117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:50.523700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-8f18a039-71a2-4ecf-8028-e8bd93bd7117 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:50.524937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8f18a039-71a2-4ecf-8028-e8bd93bd7117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:50.533006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1099/2200] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:50.537429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ee2d4-51fc-4717-8286-f5accd13cd54 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:50.547505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ee2d4-51fc-4717-8286-f5accd13cd54 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:50.552457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:50.557429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:50.557429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:50.557429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:50.557429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:50.557429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:50.557429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:50.569546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1102/2201] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:55:50.573858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18a039-71a2-4ecf-8028-e8bd93bd7117 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:50.584181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18a039-71a2-4ecf-8028-e8bd93bd7117 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:50.588971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b9614-7d74-47b0-8e5f-ee7515608bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:50.591853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b9614-7d74-47b0-8e5f-ee7515608bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:50.591853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b9614-7d74-47b0-8e5f-ee7515608bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:50.591853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b9614-7d74-47b0-8e5f-ee7515608bc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:50.591853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b9614-7d74-47b0-8e5f-ee7515608bc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:50.593004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b9614-7d74-47b0-8e5f-ee7515608bc5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:50.715077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1100/2202] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:50 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:50.719519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:50.730120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62b04e-98cd-4561-99b9-f5ffc0c14d8a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:50.759540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:50.790841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4f973-fb18-4acc-8cdc-df1d993541a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:50.791457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f973-fb18-4acc-8cdc-df1d993541a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:50.791764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f973-fb18-4acc-8cdc-df1d993541a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:50.792041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f973-fb18-4acc-8cdc-df1d993541a3 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:50.792696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4f973-fb18-4acc-8cdc-df1d993541a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:50.794688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4f973-fb18-4acc-8cdc-df1d993541a3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:50.915705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1103/2203] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:50 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:50.920118 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b9614-7d74-47b0-8e5f-ee7515608bc5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:50.931406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b9614-7d74-47b0-8e5f-ee7515608bc5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:50.936640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5b291-12a9-4743-b6fd-df2be2f9cc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:50.937270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5b291-12a9-4743-b6fd-df2be2f9cc69 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:50.937547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5b291-12a9-4743-b6fd-df2be2f9cc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:50.937884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5b291-12a9-4743-b6fd-df2be2f9cc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:50.938499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5b291-12a9-4743-b6fd-df2be2f9cc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:50.940097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5b291-12a9-4743-b6fd-df2be2f9cc69 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:50.951861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1101/2204] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:50.956029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4f973-fb18-4acc-8cdc-df1d993541a3 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:50.968060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4f973-fb18-4acc-8cdc-df1d993541a3 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:50.973476 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e4352-7f5a-4f7e-9f9f-0c975783ec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:50.974122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e4352-7f5a-4f7e-9f9f-0c975783ec62 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:50.974529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e4352-7f5a-4f7e-9f9f-0c975783ec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:50.974788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e4352-7f5a-4f7e-9f9f-0c975783ec62 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:50.975470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e4352-7f5a-4f7e-9f9f-0c975783ec62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:50.977236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e4352-7f5a-4f7e-9f9f-0c975783ec62 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:50.990019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1104/2205] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:50.994275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5b291-12a9-4743-b6fd-df2be2f9cc69 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:51.007131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5b291-12a9-4743-b6fd-df2be2f9cc69 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:51.011132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f61d4-d82a-400a-b270-9e7117decd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:51.012050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f61d4-d82a-400a-b270-9e7117decd08 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.012050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f61d4-d82a-400a-b270-9e7117decd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.014330 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f61d4-d82a-400a-b270-9e7117decd08 admin admin] PATH_INFO: `/v3/projects/034533fbec6446be9d2c155a37f728bb/users/147af6ce0a75415d9638e7aae5cb12c0/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.043245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f61d4-d82a-400a-b270-9e7117decd08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=034533fbec6446be9d2c155a37f728bb, user_id=147af6ce0a75415d9638e7aae5cb12c0, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:51.046432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f61d4-d82a-400a-b270-9e7117decd08 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:51.109014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1102/2206] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:50 2022] PUT /identity/v3/projects/034533fbec6446be9d2c155a37f728bb/users/147af6ce0a75415d9638e7aae5cb12c0/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:51.121311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d84e49-9c39-4561-ac1f-0f9f882cc200 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.121601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d84e49-9c39-4561-ac1f-0f9f882cc200 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.121873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d84e49-9c39-4561-ac1f-0f9f882cc200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.151060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-d4d84e49-9c39-4561-ac1f-0f9f882cc200 None None] MFA Rules not processed for user `147af6ce0a75415d9638e7aae5cb12c0`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:51.152369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d4d84e49-9c39-4561-ac1f-0f9f882cc200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:51.222660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1105/2207] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:51.227465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f61d4-d82a-400a-b270-9e7117decd08 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:51.271192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:51.294064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f61d4-d82a-400a-b270-9e7117decd08 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:51.299161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb6b9e-5a41-4e74-8a94-1d294717a3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:51.299790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb6b9e-5a41-4e74-8a94-1d294717a3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.300069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb6b9e-5a41-4e74-8a94-1d294717a3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.300506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb6b9e-5a41-4e74-8a94-1d294717a3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.301249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb6b9e-5a41-4e74-8a94-1d294717a3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:51.303792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb6b9e-5a41-4e74-8a94-1d294717a3f9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:51.425349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1103/2208] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:51 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:51.430116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d84e49-9c39-4561-ac1f-0f9f882cc200 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:51.440881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d84e49-9c39-4561-ac1f-0f9f882cc200 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:51.500191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3a3a3-4038-4783-9cf2-9da38a2a3c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:51.500837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3a3a3-4038-4783-9cf2-9da38a2a3c78 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.501074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3a3a3-4038-4783-9cf2-9da38a2a3c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.501348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3a3a3-4038-4783-9cf2-9da38a2a3c78 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.501981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3a3a3-4038-4783-9cf2-9da38a2a3c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:51.503950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3a3a3-4038-4783-9cf2-9da38a2a3c78 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:51.606819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1106/2209] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:51 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:55:51.611098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb6b9e-5a41-4e74-8a94-1d294717a3f9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:51.621976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb6b9e-5a41-4e74-8a94-1d294717a3f9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:51.627313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8822b7e-e687-4db6-bf8e-161d110891c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:51.627884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822b7e-e687-4db6-bf8e-161d110891c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.628161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822b7e-e687-4db6-bf8e-161d110891c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.628435 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822b7e-e687-4db6-bf8e-161d110891c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.629084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8822b7e-e687-4db6-bf8e-161d110891c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:51.630749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8822b7e-e687-4db6-bf8e-161d110891c3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:51.641995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1104/2210] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:51.647230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3a3a3-4038-4783-9cf2-9da38a2a3c78 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:51.657486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3a3a3-4038-4783-9cf2-9da38a2a3c78 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:51.663338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d301b-ef24-4c73-95b5-375958464fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:51.664009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d301b-ef24-4c73-95b5-375958464fce admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.664291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d301b-ef24-4c73-95b5-375958464fce admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.664567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d301b-ef24-4c73-95b5-375958464fce admin admin] PATH_INFO: `/v3/projects/948d8ca58c31452ab1cf234be1ed4620/users/ec425155e8d24a53ab848542b80b07db/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.681668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d301b-ef24-4c73-95b5-375958464fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=948d8ca58c31452ab1cf234be1ed4620, user_id=ec425155e8d24a53ab848542b80b07db, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:51.684892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d301b-ef24-4c73-95b5-375958464fce admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:51.778280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1107/2211] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:51 2022] PUT /identity/v3/projects/948d8ca58c31452ab1cf234be1ed4620/users/ec425155e8d24a53ab848542b80b07db/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:51.782280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8822b7e-e687-4db6-bf8e-161d110891c3 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:51.798753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8822b7e-e687-4db6-bf8e-161d110891c3 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:51.855803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3f8f5-e90d-491a-abb4-ab6aba544a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:51.856400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3f8f5-e90d-491a-abb4-ab6aba544a9d admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.856683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3f8f5-e90d-491a-abb4-ab6aba544a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.856949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3f8f5-e90d-491a-abb4-ab6aba544a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.857690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3f8f5-e90d-491a-abb4-ab6aba544a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:51.859686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3f8f5-e90d-491a-abb4-ab6aba544a9d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:51.872107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1105/2212] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:55:51.876945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d301b-ef24-4c73-95b5-375958464fce admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:51.883025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:51.888391 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d301b-ef24-4c73-95b5-375958464fce admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:51.894579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:51.894579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.895714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.895895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] PATH_INFO: `/v3/projects/948d8ca58c31452ab1cf234be1ed4620/users/ec425155e8d24a53ab848542b80b07db/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.898110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948d8ca58c31452ab1cf234be1ed4620, user_id=ec425155e8d24a53ab848542b80b07db, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:51.900904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:51.923722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1108/2213] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:51 2022] PUT /identity/v3/projects/948d8ca58c31452ab1cf234be1ed4620/users/ec425155e8d24a53ab848542b80b07db/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:51.934126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ad939-a1b4-4b21-8889-6589e2b68005 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:51.934667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ad939-a1b4-4b21-8889-6589e2b68005 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:51.934947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ad939-a1b4-4b21-8889-6589e2b68005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:51.966096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-181ad939-a1b4-4b21-8889-6589e2b68005 None None] MFA Rules not processed for user `ec425155e8d24a53ab848542b80b07db`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:51.967459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-181ad939-a1b4-4b21-8889-6589e2b68005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:52.064821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1106/2214] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:52.080977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:52.083720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:52.155142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e193c-20df-4cab-9d9e-fc70bfc04f33 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:52.159396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca19e8f-9671-42a7-ae1e-0177b8fd00cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:52.160250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca19e8f-9671-42a7-ae1e-0177b8fd00cf service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:52.160845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca19e8f-9671-42a7-ae1e-0177b8fd00cf service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:52.161412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca19e8f-9671-42a7-ae1e-0177b8fd00cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:52.178517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca19e8f-9671-42a7-ae1e-0177b8fd00cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:52.178517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca19e8f-9671-42a7-ae1e-0177b8fd00cf service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:52.189621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1109/2215] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:52.528578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ad939-a1b4-4b21-8889-6589e2b68005 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:52.539982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ad939-a1b4-4b21-8889-6589e2b68005 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:52.550362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cccde-e19b-4a8e-b629-6d5d0623ba04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:52.550362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cccde-e19b-4a8e-b629-6d5d0623ba04 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:52.550362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cccde-e19b-4a8e-b629-6d5d0623ba04 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:52.550362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cccde-e19b-4a8e-b629-6d5d0623ba04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:52.550828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8c3cccde-e19b-4a8e-b629-6d5d0623ba04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:52.587067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:52.627343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cccde-e19b-4a8e-b629-6d5d0623ba04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:52.627343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cccde-e19b-4a8e-b629-6d5d0623ba04 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:52.646311 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1107/2216] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:54.268167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca19e8f-9671-42a7-ae1e-0177b8fd00cf service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:54.367128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca19e8f-9671-42a7-ae1e-0177b8fd00cf service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:54.375959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:54.378594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:54.379479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:54.380386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] PATH_INFO: `/v3/users/147af6ce0a75415d9638e7aae5cb12c0` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:54.387594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=147af6ce0a75415d9638e7aae5cb12c0)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:54.392999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:54.703852 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '147af6ce0a75415d9638e7aae5cb12c0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:54.840213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '147af6ce0a75415d9638e7aae5cb12c0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:54.840841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '147af6ce0a75415d9638e7aae5cb12c0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:54.841459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '147af6ce0a75415d9638e7aae5cb12c0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:54.849885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '147af6ce0a75415d9638e7aae5cb12c0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:54.868888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1110/2217] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:54 2022] DELETE /identity/v3/users/147af6ce0a75415d9638e7aae5cb12c0 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:54.876798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44c68f-d6fe-4b4f-ac10-73c17edebe22 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:54.876798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44c68f-d6fe-4b4f-ac10-73c17edebe22 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:54.877646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44c68f-d6fe-4b4f-ac10-73c17edebe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:54.900042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ea44c68f-d6fe-4b4f-ac10-73c17edebe22 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:54.901276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ea44c68f-d6fe-4b4f-ac10-73c17edebe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:55.070351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1108/2218] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:55.129253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:55.129253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:55.151512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:55.209920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827ea57-7ef5-48ed-9d24-566da3f1b271 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:55.213564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be434-5b1f-4edc-b93c-08df81564915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:55.214204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be434-5b1f-4edc-b93c-08df81564915 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:55.214485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be434-5b1f-4edc-b93c-08df81564915 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:55.214756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be434-5b1f-4edc-b93c-08df81564915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:55.224038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1be434-5b1f-4edc-b93c-08df81564915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:55.225475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1be434-5b1f-4edc-b93c-08df81564915 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:55.235546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1111/2219] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:55.791607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44c68f-d6fe-4b4f-ac10-73c17edebe22 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:55.793273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ea44c68f-d6fe-4b4f-ac10-73c17edebe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:55.822543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44c68f-d6fe-4b4f-ac10-73c17edebe22 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:55.823306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:55.831832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:55.832131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:55.832415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] PATH_INFO: `/v3/projects/034533fbec6446be9d2c155a37f728bb` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:55.833804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=034533fbec6446be9d2c155a37f728bb)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:55.837009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:55.978706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '034533fbec6446be9d2c155a37f728bb', 'request_id': 'req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:55.986323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '034533fbec6446be9d2c155a37f728bb', 'request_id': 'req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.057917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '034533fbec6446be9d2c155a37f728bb', 'request_id': 'req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.074272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] The token cache is being invalidate because project 034533fbec6446be9d2c155a37f728bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:56.074712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.075721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.078125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1109/2220] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:55 2022] DELETE /identity/v3/projects/034533fbec6446be9d2c155a37f728bb => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:56.082423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be434-5b1f-4edc-b93c-08df81564915 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:56.093367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be434-5b1f-4edc-b93c-08df81564915 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:56.193321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:56.194226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:56.194673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:56.195112 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] PATH_INFO: `/v3/users/ec425155e8d24a53ab848542b80b07db` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:56.202331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec425155e8d24a53ab848542b80b07db)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:56.202331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:56.491511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec425155e8d24a53ab848542b80b07db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.642377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec425155e8d24a53ab848542b80b07db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.648132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec425155e8d24a53ab848542b80b07db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.649221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec425155e8d24a53ab848542b80b07db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.662432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a2c66c67-42a5-48f2-b0eb-67548dca4abb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec425155e8d24a53ab848542b80b07db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:56.683586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1112/2221] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:56 2022] DELETE /identity/v3/users/ec425155e8d24a53ab848542b80b07db => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:57.195769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:57.197568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:57.212058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3205-649f-4c97-8d6f-b2a4f938a4dc admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:57.281765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:57.292110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:57.295962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.295962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.295962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] PATH_INFO: `/v3/projects/948d8ca58c31452ab1cf234be1ed4620` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.295962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=948d8ca58c31452ab1cf234be1ed4620)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:57.296960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:57.494368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948d8ca58c31452ab1cf234be1ed4620', 'request_id': 'req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:57.501313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948d8ca58c31452ab1cf234be1ed4620', 'request_id': 'req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:57.611629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948d8ca58c31452ab1cf234be1ed4620', 'request_id': 'req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:57.623689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] The token cache is being invalidate because project 948d8ca58c31452ab1cf234be1ed4620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:57.623983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:57.624740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-db10eba4-cb3c-4b11-8bcb-ffe72fc362a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:57.626309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1110/2222] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:57 2022] DELETE /identity/v3/projects/948d8ca58c31452ab1cf234be1ed4620 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:57.632703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02a02-16af-4387-8d13-c15ccdd79271 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.633213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02a02-16af-4387-8d13-c15ccdd79271 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.633432 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02a02-16af-4387-8d13-c15ccdd79271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.656262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-75b02a02-16af-4387-8d13-c15ccdd79271 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:57.657421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-75b02a02-16af-4387-8d13-c15ccdd79271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:57.689003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:57.720663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1113/2223] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:57.728668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db4dae-015a-413c-b9ea-ad0ae109cb7a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.728954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db4dae-015a-413c-b9ea-ad0ae109cb7a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.729274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db4dae-015a-413c-b9ea-ad0ae109cb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.750447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c8db4dae-015a-413c-b9ea-ad0ae109cb7a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:57.751650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c8db4dae-015a-413c-b9ea-ad0ae109cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:57.769699 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1111/2224] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:57.769860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b02a02-16af-4387-8d13-c15ccdd79271 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:57.777288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b02a02-16af-4387-8d13-c15ccdd79271 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:57.782437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:57.783061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.783351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.783625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.784284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:55:57.784591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:57.786276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:57.805593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1114/2225] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:55:57.813756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db4dae-015a-413c-b9ea-ad0ae109cb7a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:57.824841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db4dae-015a-413c-b9ea-ad0ae109cb7a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:57.832740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123a5b1-b19b-4563-9b9b-90eda63875b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:57.832740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123a5b1-b19b-4563-9b9b-90eda63875b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.832740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123a5b1-b19b-4563-9b9b-90eda63875b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.832740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123a5b1-b19b-4563-9b9b-90eda63875b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.838562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123a5b1-b19b-4563-9b9b-90eda63875b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:57.838562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123a5b1-b19b-4563-9b9b-90eda63875b1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:57.881754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:57.883589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:57.898074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36036-9b4b-4099-983d-e343dcb331fd admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:57.903185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:57.903829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.904119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.904397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] PATH_INFO: `/v3/users/410ea99c613b46e28aee277eebcf56cc` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.905556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=410ea99c613b46e28aee277eebcf56cc)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:57.913288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:57.941823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1112/2226] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:55:57 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:57.944748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123a5b1-b19b-4563-9b9b-90eda63875b1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:58.019386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123a5b1-b19b-4563-9b9b-90eda63875b1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:58.026013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a88e61-9ef9-4ff6-b12c-ebdc4fc9b9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:58.026735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a88e61-9ef9-4ff6-b12c-ebdc4fc9b9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.027038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a88e61-9ef9-4ff6-b12c-ebdc4fc9b9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.027360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a88e61-9ef9-4ff6-b12c-ebdc4fc9b9d0 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.028040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a88e61-9ef9-4ff6-b12c-ebdc4fc9b9d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:58.030094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a88e61-9ef9-4ff6-b12c-ebdc4fc9b9d0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:58.202505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1113/2227] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:55:57 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:55:58.206511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a88e61-9ef9-4ff6-b12c-ebdc4fc9b9d0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:58.217666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a88e61-9ef9-4ff6-b12c-ebdc4fc9b9d0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:58.223098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '410ea99c613b46e28aee277eebcf56cc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:58.224536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72290892-862a-4d74-b34b-5915589d654f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:58.225529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72290892-862a-4d74-b34b-5915589d654f admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.225529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72290892-862a-4d74-b34b-5915589d654f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.225803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72290892-862a-4d74-b34b-5915589d654f admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.226552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72290892-862a-4d74-b34b-5915589d654f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:58.228242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72290892-862a-4d74-b34b-5915589d654f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:58.239780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1114/2228] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:58.243719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72290892-862a-4d74-b34b-5915589d654f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:58.253962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72290892-862a-4d74-b34b-5915589d654f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:58.258708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be81a0-09ad-4fac-8811-6e670a0edcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:58.259398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be81a0-09ad-4fac-8811-6e670a0edcce admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.260084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be81a0-09ad-4fac-8811-6e670a0edcce admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.260297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be81a0-09ad-4fac-8811-6e670a0edcce admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.260938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be81a0-09ad-4fac-8811-6e670a0edcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:58.262721 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be81a0-09ad-4fac-8811-6e670a0edcce admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:58.275488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1115/2229] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:55:58.280275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be81a0-09ad-4fac-8811-6e670a0edcce admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:58.291896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '410ea99c613b46e28aee277eebcf56cc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:58.292622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '410ea99c613b46e28aee277eebcf56cc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:58.293265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '410ea99c613b46e28aee277eebcf56cc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:58.304397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eac3c79f-076f-4834-b237-bc029181de8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '410ea99c613b46e28aee277eebcf56cc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:58.304547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be81a0-09ad-4fac-8811-6e670a0edcce admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:58.314130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8f750-f189-48af-8185-92155dc768fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:58.315419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f750-f189-48af-8185-92155dc768fa admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.315419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f750-f189-48af-8185-92155dc768fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.315419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f750-f189-48af-8185-92155dc768fa admin admin] PATH_INFO: `/v3/projects/fec4dc5259fd42f9857c24c060cd77f6/users/9dfda85382c44e9495752e212198a711/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.329945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1115/2230] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:57 2022] DELETE /identity/v3/users/410ea99c613b46e28aee277eebcf56cc => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:58.333948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8f750-f189-48af-8185-92155dc768fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec4dc5259fd42f9857c24c060cd77f6, user_id=9dfda85382c44e9495752e212198a711, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:58.335360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92443452-49af-44b3-a555-da2ea0e32500 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.336239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92443452-49af-44b3-a555-da2ea0e32500 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.336533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92443452-49af-44b3-a555-da2ea0e32500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.337184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8f750-f189-48af-8185-92155dc768fa admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:58.361051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-92443452-49af-44b3-a555-da2ea0e32500 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:58.362457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-92443452-49af-44b3-a555-da2ea0e32500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:58.434195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1116/2231] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:58.444765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92443452-49af-44b3-a555-da2ea0e32500 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:58.446207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-92443452-49af-44b3-a555-da2ea0e32500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:58.464456 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:58.470827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1116/2232] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:55:58 2022] PUT /identity/v3/projects/fec4dc5259fd42f9857c24c060cd77f6/users/9dfda85382c44e9495752e212198a711/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:58.486637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd284c-f60c-461d-bb3e-f1b83c124793 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.487008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd284c-f60c-461d-bb3e-f1b83c124793 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.487300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd284c-f60c-461d-bb3e-f1b83c124793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.524654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e7bd284c-f60c-461d-bb3e-f1b83c124793 None None] MFA Rules not processed for user `9dfda85382c44e9495752e212198a711`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:58.526048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e7bd284c-f60c-461d-bb3e-f1b83c124793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:58.543486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92443452-49af-44b3-a555-da2ea0e32500 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:58.608384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:58.608981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.609265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.609545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.614028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1117/2233] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:58.683899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:58.683899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:58.692917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1117/2234] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:58.701462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd284c-f60c-461d-bb3e-f1b83c124793 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:58.703779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e7bd284c-f60c-461d-bb3e-f1b83c124793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:58.779958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd284c-f60c-461d-bb3e-f1b83c124793 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:58.784109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cdebd-e152-4194-8dae-28806075e6ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:58.784750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cdebd-e152-4194-8dae-28806075e6ac service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.785029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cdebd-e152-4194-8dae-28806075e6ac service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.785408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cdebd-e152-4194-8dae-28806075e6ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.794788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cdebd-e152-4194-8dae-28806075e6ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:58.796320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cdebd-e152-4194-8dae-28806075e6ac service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:58.805819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1118/2235] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:58.839765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:58.841479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:58.859256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69018c-454c-4dc8-8381-1b51792a7015 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:58.866601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:58.867592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:58.868095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:58.868954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] PATH_INFO: `/v3/users/9dfda85382c44e9495752e212198a711` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:58.870461 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfda85382c44e9495752e212198a711)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:58.873159 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:58.938344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:59.121815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfda85382c44e9495752e212198a711'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.236405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfda85382c44e9495752e212198a711'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.237050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfda85382c44e9495752e212198a711'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.240426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfda85382c44e9495752e212198a711'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.252592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfda85382c44e9495752e212198a711'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.282743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1118/2236] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:58 2022] DELETE /identity/v3/users/9dfda85382c44e9495752e212198a711 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:55:59.289352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc9580-9e59-44fd-a42a-38a1965040c7 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:59.290019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc9580-9e59-44fd-a42a-38a1965040c7 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:59.290444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc9580-9e59-44fd-a42a-38a1965040c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:59.311492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-bcbc9580-9e59-44fd-a42a-38a1965040c7 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:59.312783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bcbc9580-9e59-44fd-a42a-38a1965040c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:59.340917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:59.342751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:59.435790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1119/2237] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:59.452428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc9580-9e59-44fd-a42a-38a1965040c7 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:59.452428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bcbc9580-9e59-44fd-a42a-38a1965040c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:59.477193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef06137-a07e-438a-9180-a30cc093c8e0 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:59.482690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:59.490167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:59.490167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:59.490167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] PATH_INFO: `/v3/projects/1752346a8fc94ef484548294dc9704ef` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:59.490167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1752346a8fc94ef484548294dc9704ef)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:59.490167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:59.545852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc9580-9e59-44fd-a42a-38a1965040c7 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:59.550506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87739a44-3bfc-4f48-ba72-f00f47f5a1eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:59.552055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87739a44-3bfc-4f48-ba72-f00f47f5a1eb service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:59.552351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87739a44-3bfc-4f48-ba72-f00f47f5a1eb service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:59.552658 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87739a44-3bfc-4f48-ba72-f00f47f5a1eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:59.562958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87739a44-3bfc-4f48-ba72-f00f47f5a1eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:59.564458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87739a44-3bfc-4f48-ba72-f00f47f5a1eb service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:59.575273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1120/2238] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:59.642275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1752346a8fc94ef484548294dc9704ef', 'request_id': 'req-75a470f9-6928-49de-b541-f7cac5c55f32'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.650880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1752346a8fc94ef484548294dc9704ef', 'request_id': 'req-75a470f9-6928-49de-b541-f7cac5c55f32'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.758448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1752346a8fc94ef484548294dc9704ef', 'request_id': 'req-75a470f9-6928-49de-b541-f7cac5c55f32'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.769316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] The token cache is being invalidate because project 1752346a8fc94ef484548294dc9704ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:55:59.769608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.770874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-75a470f9-6928-49de-b541-f7cac5c55f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:55:59.772333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1119/2239] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:55:59 2022] DELETE /identity/v3/projects/1752346a8fc94ef484548294dc9704ef => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:55:59.776569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87739a44-3bfc-4f48-ba72-f00f47f5a1eb service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:59.787116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87739a44-3bfc-4f48-ba72-f00f47f5a1eb service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:59.846882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:59.847673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:59.847971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:59.848259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] PATH_INFO: `/v3/users/e8ae2af8ea684ca9a9deb62ea3cebb58` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:59.849975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ae2af8ea684ca9a9deb62ea3cebb58)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:59.851947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:00.032022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ae2af8ea684ca9a9deb62ea3cebb58'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:00.133490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ae2af8ea684ca9a9deb62ea3cebb58'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:00.134398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ae2af8ea684ca9a9deb62ea3cebb58'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:00.154485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ae2af8ea684ca9a9deb62ea3cebb58'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:00.163411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ae2af8ea684ca9a9deb62ea3cebb58'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:00.174926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1121/2240] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:55:59 2022] DELETE /identity/v3/users/e8ae2af8ea684ca9a9deb62ea3cebb58 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:56:00.236191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7aa85-f745-4a88-9a16-17978afc2cbd None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:00.236487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7aa85-f745-4a88-9a16-17978afc2cbd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:00.236783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7aa85-f745-4a88-9a16-17978afc2cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:00.256656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b3b7aa85-f745-4a88-9a16-17978afc2cbd None None] MFA Rules not processed for user `792808dcbf894aa98c671cf43ac21b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:00.257812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b3b7aa85-f745-4a88-9a16-17978afc2cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:00.317656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1120/2241] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:00.349558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:00.366019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:00.367569 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:00.531249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7aa85-f745-4a88-9a16-17978afc2cbd None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:00.534355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b3b7aa85-f745-4a88-9a16-17978afc2cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:00.542646 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c157-2a88-439a-afa8-965349ab4cb4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:00.566896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:00.567682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:00.568192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:00.568425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7aa85-f745-4a88-9a16-17978afc2cbd None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:00.568661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] PATH_INFO: `/v3/projects/fec4dc5259fd42f9857c24c060cd77f6` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:00.577508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec4dc5259fd42f9857c24c060cd77f6)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:00.582720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:00.583562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:00.584033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:00.584472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] PATH_INFO: `/v3/users/985b73bbf878487abd05344d6a304c7e` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:00.586047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:00.586465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=985b73bbf878487abd05344d6a304c7e)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:00.589795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:00.894050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985b73bbf878487abd05344d6a304c7e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:00.900732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:00.913009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec4dc5259fd42f9857c24c060cd77f6', 'request_id': 'req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:00.920515 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec4dc5259fd42f9857c24c060cd77f6', 'request_id': 'req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.023678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985b73bbf878487abd05344d6a304c7e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.024412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:01.024539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985b73bbf878487abd05344d6a304c7e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.025253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985b73bbf878487abd05344d6a304c7e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.041850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985b73bbf878487abd05344d6a304c7e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.076574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1121/2242] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:00 2022] DELETE /identity/v3/users/985b73bbf878487abd05344d6a304c7e => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:01.077833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:01.082174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:01.159757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c0e8b-a9c1-48c3-883f-d56cac988997 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:01.165366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:01.166016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.168853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.168853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] PATH_INFO: `/v3/projects/179adadc4935466d836ae89ab11fd634` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.168853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=179adadc4935466d836ae89ab11fd634)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:01.169874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:01.200008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec4dc5259fd42f9857c24c060cd77f6', 'request_id': 'req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.211106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] The token cache is being invalidate because project fec4dc5259fd42f9857c24c060cd77f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:01.211407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.212087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9b5378aa-9fe9-4443-ad9c-5fa782a3431a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.213582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1122/2243] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:00 2022] DELETE /identity/v3/projects/fec4dc5259fd42f9857c24c060cd77f6 => generated 0 bytes in 848 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:01.214793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a064d-9612-4e89-b4b1-d2700cd48a10 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.215078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a064d-9612-4e89-b4b1-d2700cd48a10 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.215353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a064d-9612-4e89-b4b1-d2700cd48a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.248159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-2b7a064d-9612-4e89-b4b1-d2700cd48a10 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:01.249473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2b7a064d-9612-4e89-b4b1-d2700cd48a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:01.258184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1123/2244] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:01.262205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d8e9-663f-4f20-8903-978b5b37649f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.262205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d8e9-663f-4f20-8903-978b5b37649f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.262205 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d8e9-663f-4f20-8903-978b5b37649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.295394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-9662d8e9-663f-4f20-8903-978b5b37649f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:01.295394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9662d8e9-663f-4f20-8903-978b5b37649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:01.329248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179adadc4935466d836ae89ab11fd634', 'request_id': 'req-6f48dc3b-960d-4756-a53a-26751ce567ec'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.344825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179adadc4935466d836ae89ab11fd634', 'request_id': 'req-6f48dc3b-960d-4756-a53a-26751ce567ec'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.389458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1124/2245] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:01.390302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662d8e9-663f-4f20-8903-978b5b37649f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:01.391785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9662d8e9-663f-4f20-8903-978b5b37649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:01.414201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179adadc4935466d836ae89ab11fd634', 'request_id': 'req-6f48dc3b-960d-4756-a53a-26751ce567ec'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.430942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] The token cache is being invalidate because project 179adadc4935466d836ae89ab11fd634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:01.431247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.431980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-6f48dc3b-960d-4756-a53a-26751ce567ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:01.434999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1122/2246] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:01 2022] DELETE /identity/v3/projects/179adadc4935466d836ae89ab11fd634 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:01.440538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d606b9a-c77f-47c3-93ee-d09b83e69af0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.440701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d606b9a-c77f-47c3-93ee-d09b83e69af0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.440982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d606b9a-c77f-47c3-93ee-d09b83e69af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.477874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3d606b9a-c77f-47c3-93ee-d09b83e69af0 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:01.480322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3d606b9a-c77f-47c3-93ee-d09b83e69af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:01.489041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662d8e9-663f-4f20-8903-978b5b37649f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:01.491853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1123/2247] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:01.492686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d606b9a-c77f-47c3-93ee-d09b83e69af0 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:01.495578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3d606b9a-c77f-47c3-93ee-d09b83e69af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:01.495890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:01.496595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.496941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.497280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.499650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:01.508657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d606b9a-c77f-47c3-93ee-d09b83e69af0 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:01.516785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:01.517541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.517861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.518266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] PATH_INFO: `/v3/users/0efd382e03664e58acdf004b120d506c` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.519772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efd382e03664e58acdf004b120d506c)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:01.522663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:01.525529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:01.528531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:01.546786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1125/2248] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:01.547345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:01.557570 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad4a4-e0a5-48b3-8a30-463a8e9d7c67 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:01.568410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:01.569331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.569811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.570223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.570886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:01.571225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:01.572909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:01.588516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1126/2249] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:56:01.594059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:01.605153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1290f7b-dfb6-4e0d-bb71-4e84c74bbbf7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:01.610023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626741f-9848-4a0f-a731-04e6044cc0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:01.610667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626741f-9848-4a0f-a731-04e6044cc0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.610951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626741f-9848-4a0f-a731-04e6044cc0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.611234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626741f-9848-4a0f-a731-04e6044cc0e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.611862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626741f-9848-4a0f-a731-04e6044cc0e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:01.613982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626741f-9848-4a0f-a731-04e6044cc0e2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:01.883355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1127/2250] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:01 2022] POST /identity/v3/projects => generated 373 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:01.887437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626741f-9848-4a0f-a731-04e6044cc0e2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.002075 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efd382e03664e58acdf004b120d506c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.002985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626741f-9848-4a0f-a731-04e6044cc0e2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.011684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.012735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.013251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.014050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.016415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.020105 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.104260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efd382e03664e58acdf004b120d506c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.105003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efd382e03664e58acdf004b120d506c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.111287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efd382e03664e58acdf004b120d506c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.125253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efd382e03664e58acdf004b120d506c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.133249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1128/2251] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:01 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:02.138022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.139532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:02.149422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1124/2252] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:01 2022] DELETE /identity/v3/users/0efd382e03664e58acdf004b120d506c => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:02.153074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207dc4-871a-47be-9a43-b85600682fa6 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.222165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.222816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.223100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.223406 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.224060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.225677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.237685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1129/2253] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:56:02.243224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.253590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572f028-3b97-48f6-a821-6f2a6b62a59e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.258939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781a4fd-e15a-4bc6-ba34-1025e7141abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.259811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781a4fd-e15a-4bc6-ba34-1025e7141abb admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.260094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781a4fd-e15a-4bc6-ba34-1025e7141abb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.260364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781a4fd-e15a-4bc6-ba34-1025e7141abb admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.261022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781a4fd-e15a-4bc6-ba34-1025e7141abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.262728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781a4fd-e15a-4bc6-ba34-1025e7141abb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.275248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1125/2254] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:02.275777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.280389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781a4fd-e15a-4bc6-ba34-1025e7141abb admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.281102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:02.293231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781a4fd-e15a-4bc6-ba34-1025e7141abb admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.296881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c4289-af6d-4a58-9d4b-770d92812796 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.299601 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554282a-c6a4-49fd-aae7-07c5296247cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.300441 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554282a-c6a4-49fd-aae7-07c5296247cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.302229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554282a-c6a4-49fd-aae7-07c5296247cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.302621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.303193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554282a-c6a4-49fd-aae7-07c5296247cb admin admin] PATH_INFO: `/v3/projects/5be47c3b2c4c4fc2a1db117433cbcfc4/users/f2026170f4674b63a37534c14689bd10/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.303313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.303546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.303828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] PATH_INFO: `/v3/projects/e62a77dc2ac8432d93b94431f807ec15` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.305093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e62a77dc2ac8432d93b94431f807ec15)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.307749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.324271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554282a-c6a4-49fd-aae7-07c5296247cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be47c3b2c4c4fc2a1db117433cbcfc4, user_id=f2026170f4674b63a37534c14689bd10, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.327870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554282a-c6a4-49fd-aae7-07c5296247cb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.433332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1126/2255] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:02 2022] PUT /identity/v3/projects/5be47c3b2c4c4fc2a1db117433cbcfc4/users/f2026170f4674b63a37534c14689bd10/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:02.440220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c90170-25c4-4c7a-b242-c1ae26b051cb None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.440565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c90170-25c4-4c7a-b242-c1ae26b051cb None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.440849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c90170-25c4-4c7a-b242-c1ae26b051cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.448640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62a77dc2ac8432d93b94431f807ec15', 'request_id': 'req-a89250d9-00ff-448f-8917-a837912780a8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.479689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62a77dc2ac8432d93b94431f807ec15', 'request_id': 'req-a89250d9-00ff-448f-8917-a837912780a8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.490835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-46c90170-25c4-4c7a-b242-c1ae26b051cb None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:02.492156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-46c90170-25c4-4c7a-b242-c1ae26b051cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:02.596667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62a77dc2ac8432d93b94431f807ec15', 'request_id': 'req-a89250d9-00ff-448f-8917-a837912780a8'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.610877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1127/2256] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:02.623663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c90170-25c4-4c7a-b242-c1ae26b051cb None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.624120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] The token cache is being invalidate because project e62a77dc2ac8432d93b94431f807ec15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:02.624691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.625876 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a89250d9-00ff-448f-8917-a837912780a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:02.628137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1130/2257] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:02 2022] DELETE /identity/v3/projects/e62a77dc2ac8432d93b94431f807ec15 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:56:02.631936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-46c90170-25c4-4c7a-b242-c1ae26b051cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:02.633540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4042fe-0e03-49ee-9fad-b78d0fc029d8 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.634398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4042fe-0e03-49ee-9fad-b78d0fc029d8 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.634910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4042fe-0e03-49ee-9fad-b78d0fc029d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.667550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1f4042fe-0e03-49ee-9fad-b78d0fc029d8 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:02.669250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1f4042fe-0e03-49ee-9fad-b78d0fc029d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:02.680395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1131/2258] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:02.692983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bed39-ec1e-4e15-9843-2fc9910011c1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.696109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bed39-ec1e-4e15-9843-2fc9910011c1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.696109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bed39-ec1e-4e15-9843-2fc9910011c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.722833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c90170-25c4-4c7a-b242-c1ae26b051cb None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.723150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-438bed39-ec1e-4e15-9843-2fc9910011c1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:02.725096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-438bed39-ec1e-4e15-9843-2fc9910011c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:02.727586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.728328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.728704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.729106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.731086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:02.736044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1132/2259] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:02.740845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bed39-ec1e-4e15-9843-2fc9910011c1 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.747466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.749378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.751611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bed39-ec1e-4e15-9843-2fc9910011c1 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.757775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.758521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.759000 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.759338 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.760066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:02.760433 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.761377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1128/2260] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:56:02.762478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.777295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1133/2261] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:56:02.782304 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.793111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbeb8f-515c-42b8-b4cc-17b8f478377b service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.799521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c83e2c-3420-40c9-8bd2-157a7faca915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.800793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83e2c-3420-40c9-8bd2-157a7faca915 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.801025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83e2c-3420-40c9-8bd2-157a7faca915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.801364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83e2c-3420-40c9-8bd2-157a7faca915 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.802065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c83e2c-3420-40c9-8bd2-157a7faca915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.804110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c83e2c-3420-40c9-8bd2-157a7faca915 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.823543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:02.825633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:02.855858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6d156-eba6-4ce7-baca-c469ad3108e5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:02.864752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:02.865864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:02.866476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:02.867104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] PATH_INFO: `/v3/projects/08eb1503034742339939ee950e7d96ef` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:02.869088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] RBAC: Authorizing `identity:delete_project(project_id=08eb1503034742339939ee950e7d96ef)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:02.872680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:02.913422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:02.943577 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1129/2262] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:02 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:02.948092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c83e2c-3420-40c9-8bd2-157a7faca915 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.029309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08eb1503034742339939ee950e7d96ef', 'request_id': 'req-95ad9072-32e1-422c-bba7-e96b7f10409d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:03.039608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08eb1503034742339939ee950e7d96ef', 'request_id': 'req-95ad9072-32e1-422c-bba7-e96b7f10409d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:03.059985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:03.063387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c83e2c-3420-40c9-8bd2-157a7faca915 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.148984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08eb1503034742339939ee950e7d96ef', 'request_id': 'req-95ad9072-32e1-422c-bba7-e96b7f10409d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:03.176105 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] The token cache is being invalidate because project 08eb1503034742339939ee950e7d96ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:03.176670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.176913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:03.178783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-95ad9072-32e1-422c-bba7-e96b7f10409d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:03.180707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1134/2263] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:02 2022] DELETE /identity/v3/projects/08eb1503034742339939ee950e7d96ef => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:03.189797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cb1b5-7d8c-4a86-a4a4-c189726bb529 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.190183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cb1b5-7d8c-4a86-a4a4-c189726bb529 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.190536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cb1b5-7d8c-4a86-a4a4-c189726bb529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.194307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.195629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.196328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.200925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.209941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.239559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f00cb1b5-7d8c-4a86-a4a4-c189726bb529 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:03.240687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f00cb1b5-7d8c-4a86-a4a4-c189726bb529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.251134 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1135/2264] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:03.259891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b5422-594f-4ccb-87c7-63f557ea455b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.260264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b5422-594f-4ccb-87c7-63f557ea455b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.260619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b5422-594f-4ccb-87c7-63f557ea455b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.293735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c71b5422-594f-4ccb-87c7-63f557ea455b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:03.295078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c71b5422-594f-4ccb-87c7-63f557ea455b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.303796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1130/2265] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:02 2022] POST /identity/v3/users => generated 289 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:56:03.308136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.308829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1136/2266] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:03.313820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b5422-594f-4ccb-87c7-63f557ea455b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.315290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.326830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b5422-594f-4ccb-87c7-63f557ea455b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.332661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e119e-99e9-496c-aa3a-a3777c6dbb7f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.337116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.337869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.338296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.338649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.339451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:03.339933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.341085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee2214-b66d-4140-bb7d-53767f100a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.341938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.342442 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee2214-b66d-4140-bb7d-53767f100a6d admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.342938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee2214-b66d-4140-bb7d-53767f100a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.343287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee2214-b66d-4140-bb7d-53767f100a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.344085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee2214-b66d-4140-bb7d-53767f100a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.345789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee2214-b66d-4140-bb7d-53767f100a6d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.357125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1137/2267] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:56:03.361600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.362457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1131/2268] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:03.364059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee2214-b66d-4140-bb7d-53767f100a6d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.375327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee2214-b66d-4140-bb7d-53767f100a6d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.375545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c3ab6-4d79-4396-820f-c9104d24623f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.382182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3107a-c658-4acf-a224-6574579deb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.382349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9dbb9-261d-47a4-bbdf-0e4ee9596284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.383084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dbb9-261d-47a4-bbdf-0e4ee9596284 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.383196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3107a-c658-4acf-a224-6574579deb7e admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.383484 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dbb9-261d-47a4-bbdf-0e4ee9596284 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.383832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9dbb9-261d-47a4-bbdf-0e4ee9596284 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.384581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3107a-c658-4acf-a224-6574579deb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.384712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9dbb9-261d-47a4-bbdf-0e4ee9596284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.384938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3107a-c658-4acf-a224-6574579deb7e admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.386714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3107a-c658-4acf-a224-6574579deb7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.386834 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9dbb9-261d-47a4-bbdf-0e4ee9596284 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.388622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3107a-c658-4acf-a224-6574579deb7e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.401325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1132/2269] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:03.410757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9dbb9-261d-47a4-bbdf-0e4ee9596284 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.423200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9dbb9-261d-47a4-bbdf-0e4ee9596284 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.428503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c95ea-9d21-4b45-a43c-94d2158e60fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.429183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c95ea-9d21-4b45-a43c-94d2158e60fe admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.429769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c95ea-9d21-4b45-a43c-94d2158e60fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.430170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c95ea-9d21-4b45-a43c-94d2158e60fe admin admin] PATH_INFO: `/v3/projects/45579d93010d4ed0b80cad3a8b85ea50/users/772205d93ce047e9ad4ddefadec300f4/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.449219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c95ea-9d21-4b45-a43c-94d2158e60fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=45579d93010d4ed0b80cad3a8b85ea50, user_id=772205d93ce047e9ad4ddefadec300f4, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.452921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c95ea-9d21-4b45-a43c-94d2158e60fe admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.535584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1138/2270] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:56:03.538838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7dd0-08cb-4c12-9b05-f1c5059f40f7 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.538838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7dd0-08cb-4c12-9b05-f1c5059f40f7 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.538838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7dd0-08cb-4c12-9b05-f1c5059f40f7 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.546460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1139/2271] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:56:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:56:03.548024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7dd0-08cb-4c12-9b05-f1c5059f40f7 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.561835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1133/2272] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:03 2022] PUT /identity/v3/projects/45579d93010d4ed0b80cad3a8b85ea50/users/772205d93ce047e9ad4ddefadec300f4/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:03.573619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a464a0a-16d9-4a3f-ab57-c6bcfdb7e526 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.574084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a464a0a-16d9-4a3f-ab57-c6bcfdb7e526 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.574543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a464a0a-16d9-4a3f-ab57-c6bcfdb7e526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.665465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-7a464a0a-16d9-4a3f-ab57-c6bcfdb7e526 None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:03.667686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7a464a0a-16d9-4a3f-ab57-c6bcfdb7e526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.702830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7dd0-08cb-4c12-9b05-f1c5059f40f7 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.709245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477d430-0fea-43be-82b9-97b189d6c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.709933 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477d430-0fea-43be-82b9-97b189d6c07c admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.710313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477d430-0fea-43be-82b9-97b189d6c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.710657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477d430-0fea-43be-82b9-97b189d6c07c admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.711496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477d430-0fea-43be-82b9-97b189d6c07c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.713833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477d430-0fea-43be-82b9-97b189d6c07c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.780013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1134/2273] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:03.781760 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b87ffd-f7ec-4e7c-9beb-45d95efecbbd None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.782258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b87ffd-f7ec-4e7c-9beb-45d95efecbbd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.782748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b87ffd-f7ec-4e7c-9beb-45d95efecbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.808800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-70b87ffd-f7ec-4e7c-9beb-45d95efecbbd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:03.810054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-70b87ffd-f7ec-4e7c-9beb-45d95efecbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.819102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1135/2274] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:03.822705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b87ffd-f7ec-4e7c-9beb-45d95efecbbd None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.823739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-70b87ffd-f7ec-4e7c-9beb-45d95efecbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.842068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b87ffd-f7ec-4e7c-9beb-45d95efecbbd None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.848163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.849438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.849773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.850052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.859195 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.860838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.872448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1136/2275] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:03.873328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.874657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.894650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1140/2276] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/users => generated 292 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:03.896851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477d430-0fea-43be-82b9-97b189d6c07c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.907660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477d430-0fea-43be-82b9-97b189d6c07c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.914540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cce10-fa6a-4f3a-ae43-e885927b7b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.915160 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cce10-fa6a-4f3a-ae43-e885927b7b43 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.915436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cce10-fa6a-4f3a-ae43-e885927b7b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.915847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cce10-fa6a-4f3a-ae43-e885927b7b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.916709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cce10-fa6a-4f3a-ae43-e885927b7b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.918428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cce10-fa6a-4f3a-ae43-e885927b7b43 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.933320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1141/2277] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:03.939255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cce10-fa6a-4f3a-ae43-e885927b7b43 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.951769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cce10-fa6a-4f3a-ae43-e885927b7b43 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.958128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d72b2e-e509-4aca-99fb-6a4f394f4ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.958387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d72b2e-e509-4aca-99fb-6a4f394f4ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.958656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d72b2e-e509-4aca-99fb-6a4f394f4ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.958974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d72b2e-e509-4aca-99fb-6a4f394f4ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.959637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d72b2e-e509-4aca-99fb-6a4f394f4ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.961310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d72b2e-e509-4aca-99fb-6a4f394f4ff0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.961707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60d12-2add-4d5c-93a5-7479036fb2de service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.966071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c253bc9-ba8e-42dd-9d8e-072e0e9db2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:03.967652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c253bc9-ba8e-42dd-9d8e-072e0e9db2cf service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.967652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c253bc9-ba8e-42dd-9d8e-072e0e9db2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.968140 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c253bc9-ba8e-42dd-9d8e-072e0e9db2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.982740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1142/2278] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:56:03.984227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c253bc9-ba8e-42dd-9d8e-072e0e9db2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:03.985348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d72b2e-e509-4aca-99fb-6a4f394f4ff0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.986050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c253bc9-ba8e-42dd-9d8e-072e0e9db2cf service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:03.998632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1137/2279] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:04.002013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d72b2e-e509-4aca-99fb-6a4f394f4ff0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.007175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cf5d-3a11-4479-822a-70d89f591bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.007910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cf5d-3a11-4479-822a-70d89f591bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.008195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cf5d-3a11-4479-822a-70d89f591bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.008517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4cf5d-3a11-4479-822a-70d89f591bd5 admin admin] PATH_INFO: `/v3/projects/fca3016ece994fa0b5146e5ff1da819c/users/7267e98b56194572bcc7e747dba96acc/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.026884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4cf5d-3a11-4479-822a-70d89f591bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca3016ece994fa0b5146e5ff1da819c, user_id=7267e98b56194572bcc7e747dba96acc, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.030262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4cf5d-3a11-4479-822a-70d89f591bd5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.139952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1143/2280] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:03 2022] PUT /identity/v3/projects/fca3016ece994fa0b5146e5ff1da819c/users/7267e98b56194572bcc7e747dba96acc/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:04.151677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0141543-ab37-475c-9cdd-fcae9e8400f1 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.152136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0141543-ab37-475c-9cdd-fcae9e8400f1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.156326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0141543-ab37-475c-9cdd-fcae9e8400f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.190723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f0141543-ab37-475c-9cdd-fcae9e8400f1 None None] MFA Rules not processed for user `7267e98b56194572bcc7e747dba96acc`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:04.190723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f0141543-ab37-475c-9cdd-fcae9e8400f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:04.272610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1138/2281] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:04.276918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cf5d-3a11-4479-822a-70d89f591bd5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.360353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4cf5d-3a11-4479-822a-70d89f591bd5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.368942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db00e7-561c-47f7-a90b-b87cc983f1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.369628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db00e7-561c-47f7-a90b-b87cc983f1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.369915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db00e7-561c-47f7-a90b-b87cc983f1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.370222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db00e7-561c-47f7-a90b-b87cc983f1d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.371180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db00e7-561c-47f7-a90b-b87cc983f1d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.372810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db00e7-561c-47f7-a90b-b87cc983f1d9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.479114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0141543-ab37-475c-9cdd-fcae9e8400f1 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.482034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f0141543-ab37-475c-9cdd-fcae9e8400f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:04.487469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1144/2282] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:04 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:04.488478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db00e7-561c-47f7-a90b-b87cc983f1d9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.512198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0141543-ab37-475c-9cdd-fcae9e8400f1 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.561661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db00e7-561c-47f7-a90b-b87cc983f1d9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.566682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f5e88-81d1-4943-b758-6c71b03073a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.567318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f5e88-81d1-4943-b758-6c71b03073a5 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.568610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f5e88-81d1-4943-b758-6c71b03073a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.568610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f5e88-81d1-4943-b758-6c71b03073a5 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.568610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f5e88-81d1-4943-b758-6c71b03073a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.570587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f5e88-81d1-4943-b758-6c71b03073a5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.574845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.575492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.575775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.576150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] PATH_INFO: `/v3/users/764a2584f5544c9d93ba471e72bd36ce` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.577404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=764a2584f5544c9d93ba471e72bd36ce)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.579403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.660663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1145/2283] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:04 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:04.663943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f5e88-81d1-4943-b758-6c71b03073a5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.690584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f5e88-81d1-4943-b758-6c71b03073a5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.701674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03316612-4c28-4488-8292-1ba2306b6383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.702634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03316612-4c28-4488-8292-1ba2306b6383 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.703264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03316612-4c28-4488-8292-1ba2306b6383 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.703807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03316612-4c28-4488-8292-1ba2306b6383 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.704948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03316612-4c28-4488-8292-1ba2306b6383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.706930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03316612-4c28-4488-8292-1ba2306b6383 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.730320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1146/2284] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:04.734739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03316612-4c28-4488-8292-1ba2306b6383 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.758361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03316612-4c28-4488-8292-1ba2306b6383 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.772893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d37fc-a144-4bac-9ca7-90015ca11de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.778370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d37fc-a144-4bac-9ca7-90015ca11de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.778927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d37fc-a144-4bac-9ca7-90015ca11de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.779235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d37fc-a144-4bac-9ca7-90015ca11de5 admin admin] PATH_INFO: `/v3/projects/7e9a8f50400c4ce3bf22c4fcd1c421b0/users/2be4382bf94347ecba98e563decad26e/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.806636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d37fc-a144-4bac-9ca7-90015ca11de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9a8f50400c4ce3bf22c4fcd1c421b0, user_id=2be4382bf94347ecba98e563decad26e, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.812964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d37fc-a144-4bac-9ca7-90015ca11de5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.903833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764a2584f5544c9d93ba471e72bd36ce'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:04.994211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1147/2285] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:04 2022] PUT /identity/v3/projects/7e9a8f50400c4ce3bf22c4fcd1c421b0/users/2be4382bf94347ecba98e563decad26e/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:04.997888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d37fc-a144-4bac-9ca7-90015ca11de5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:05.029144 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764a2584f5544c9d93ba471e72bd36ce'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.029824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764a2584f5544c9d93ba471e72bd36ce'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.030498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764a2584f5544c9d93ba471e72bd36ce'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.043674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764a2584f5544c9d93ba471e72bd36ce'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.073182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1139/2286] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:04 2022] DELETE /identity/v3/users/764a2584f5544c9d93ba471e72bd36ce => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:05.110228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:05.128984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d37fc-a144-4bac-9ca7-90015ca11de5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:05.167605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:05.297780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b5f15-7ae1-4b9e-b5e5-138dbb01c1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:05.298560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b5f15-7ae1-4b9e-b5e5-138dbb01c1dc admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.298844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b5f15-7ae1-4b9e-b5e5-138dbb01c1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.299116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b5f15-7ae1-4b9e-b5e5-138dbb01c1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.299811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b5f15-7ae1-4b9e-b5e5-138dbb01c1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:05.305735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b5f15-7ae1-4b9e-b5e5-138dbb01c1dc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:05.318060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1148/2287] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:56:05.322264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:05.323737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:05.353040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042563ab-07e1-4d79-9b7c-9038bcbe3c49 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:05.359013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:05.360059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.361183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.361762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] PATH_INFO: `/v3/projects/7e9a8f50400c4ce3bf22c4fcd1c421b0/users/2be4382bf94347ecba98e563decad26e/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.366291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9a8f50400c4ce3bf22c4fcd1c421b0, user_id=2be4382bf94347ecba98e563decad26e, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:05.370975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:05.384333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1140/2288] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:05 2022] PUT /identity/v3/projects/7e9a8f50400c4ce3bf22c4fcd1c421b0/users/2be4382bf94347ecba98e563decad26e/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:05.396307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0be0c-4203-4c50-93a4-5b655d716824 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.397826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0be0c-4203-4c50-93a4-5b655d716824 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.397826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0be0c-4203-4c50-93a4-5b655d716824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.432104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-27a0be0c-4203-4c50-93a4-5b655d716824 None None] MFA Rules not processed for user `2be4382bf94347ecba98e563decad26e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:05.433429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-27a0be0c-4203-4c50-93a4-5b655d716824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:05.513730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:05.516879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:05.576418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1149/2289] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:05.586439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0be0c-4203-4c50-93a4-5b655d716824 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:05.588684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-27a0be0c-4203-4c50-93a4-5b655d716824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:05.668610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cf5de-9cda-46e7-b8fa-0268a86e957a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:05.675469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:05.676127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.677190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.677471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] PATH_INFO: `/v3/projects/0f3172b5d47145b7bb46df4ce131dcf2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.679612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f3172b5d47145b7bb46df4ce131dcf2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:05.684908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:05.686484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0be0c-4203-4c50-93a4-5b655d716824 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:05.692859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:05.694077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.694659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.694659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.706350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:05.708030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:05.718547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1150/2290] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:56:05.858654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f3172b5d47145b7bb46df4ce131dcf2', 'request_id': 'req-74e24f59-a475-415d-b2d1-90617efe83af'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.868854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f3172b5d47145b7bb46df4ce131dcf2', 'request_id': 'req-74e24f59-a475-415d-b2d1-90617efe83af'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.912215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:05.914039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:05.927790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05538f-96e0-44e4-a5cf-0bfa7d2b2292 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:05.949138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f3172b5d47145b7bb46df4ce131dcf2', 'request_id': 'req-74e24f59-a475-415d-b2d1-90617efe83af'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.968647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] The token cache is being invalidate because project 0f3172b5d47145b7bb46df4ce131dcf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:05.969284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.970515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.972275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1141/2291] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:05 2022] DELETE /identity/v3/projects/0f3172b5d47145b7bb46df4ce131dcf2 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:56:05.984117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:05.995296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:06.072332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:06.072888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:06.073275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:06.073545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:06.078214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:06.201041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e24f59-a475-415d-b2d1-90617efe83af admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:06.215091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:06.218369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:06.219970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:06.222206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] PATH_INFO: `/v3/users/793674c65b4147b39b94bacb791e3286` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:06.224597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793674c65b4147b39b94bacb791e3286)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:06.226615 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:06.252071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:06.256025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:06.282536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1151/2292] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:06.574251 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793674c65b4147b39b94bacb791e3286'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.718900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793674c65b4147b39b94bacb791e3286'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.719589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793674c65b4147b39b94bacb791e3286'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.720187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793674c65b4147b39b94bacb791e3286'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.729896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-40764cc8-c321-43bb-8192-0318a6dd5f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793674c65b4147b39b94bacb791e3286'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.764904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1142/2293] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:05 2022] DELETE /identity/v3/users/793674c65b4147b39b94bacb791e3286 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:06.948572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:07.018467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:07.087037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:07.089289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:07.130117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cc5f-ce93-4b7b-9b2d-2e8163c2c4ef service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:07.236925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:07.237596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:07.237883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:07.238988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] PATH_INFO: `/v3/users/7267e98b56194572bcc7e747dba96acc` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:07.259810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7267e98b56194572bcc7e747dba96acc)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:07.267553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:07.530507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7267e98b56194572bcc7e747dba96acc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:07.658587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7267e98b56194572bcc7e747dba96acc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:07.659385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7267e98b56194572bcc7e747dba96acc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:07.717948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7267e98b56194572bcc7e747dba96acc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:07.733644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7267e98b56194572bcc7e747dba96acc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:07.751985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1152/2294] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:07 2022] DELETE /identity/v3/users/7267e98b56194572bcc7e747dba96acc => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:07.754418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5ce2f-7acf-4393-b818-54348a5409c5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:07.755491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5ce2f-7acf-4393-b818-54348a5409c5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:07.755491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5ce2f-7acf-4393-b818-54348a5409c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:07.797941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-02d5ce2f-7acf-4393-b818-54348a5409c5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:07.799250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-02d5ce2f-7acf-4393-b818-54348a5409c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:07.992175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1143/2295] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:08.045542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:08.048152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:08.096062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5ce2f-7acf-4393-b818-54348a5409c5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:08.098181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-02d5ce2f-7acf-4393-b818-54348a5409c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:08.123150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5ce2f-7acf-4393-b818-54348a5409c5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:08.141083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:08.141886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:08.142425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:08.142876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] PATH_INFO: `/v3/projects/417e2c4567c848b2937de6f1efbf3559` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:08.147609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=417e2c4567c848b2937de6f1efbf3559)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:08.156289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:08.297430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e1d62-3102-4e63-b2d0-1c92330305f4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:08.324997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a560696-92d7-4977-88bf-deca496247e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:08.325884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560696-92d7-4977-88bf-deca496247e9 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:08.326190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560696-92d7-4977-88bf-deca496247e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:08.326601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560696-92d7-4977-88bf-deca496247e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:08.336960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a560696-92d7-4977-88bf-deca496247e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:08.338571 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a560696-92d7-4977-88bf-deca496247e9 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:08.352864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1153/2296] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:08.379427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417e2c4567c848b2937de6f1efbf3559', 'request_id': 'req-ab39247e-4375-4516-93ea-b99a54663d80'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:08.384444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417e2c4567c848b2937de6f1efbf3559', 'request_id': 'req-ab39247e-4375-4516-93ea-b99a54663d80'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:08.463537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417e2c4567c848b2937de6f1efbf3559', 'request_id': 'req-ab39247e-4375-4516-93ea-b99a54663d80'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:08.477052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] The token cache is being invalidate because project 417e2c4567c848b2937de6f1efbf3559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:08.477349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:08.478330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ab39247e-4375-4516-93ea-b99a54663d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:08.480067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1144/2297] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:08 2022] DELETE /identity/v3/projects/417e2c4567c848b2937de6f1efbf3559 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:56:08.485341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289842a-0a01-4a24-b564-306aa1fc5071 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:08.485626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289842a-0a01-4a24-b564-306aa1fc5071 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:08.485906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289842a-0a01-4a24-b564-306aa1fc5071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:08.512928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0289842a-0a01-4a24-b564-306aa1fc5071 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:08.514484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0289842a-0a01-4a24-b564-306aa1fc5071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:08.639257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79d0f3-5539-438c-b384-b0e4faca52e5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:08.639257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79d0f3-5539-438c-b384-b0e4faca52e5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:08.639257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79d0f3-5539-438c-b384-b0e4faca52e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:08.645713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1154/2298] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:08.677200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-7f79d0f3-5539-438c-b384-b0e4faca52e5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:08.679283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7f79d0f3-5539-438c-b384-b0e4faca52e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:08.699371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289842a-0a01-4a24-b564-306aa1fc5071 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:08.708121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1145/2299] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:08.713956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289842a-0a01-4a24-b564-306aa1fc5071 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:08.723193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:08.723935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:08.724438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:08.724815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:08.725621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:08.726027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:08.729460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:08.758334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1155/2300] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:56:08.762626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79d0f3-5539-438c-b384-b0e4faca52e5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:08.791319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79d0f3-5539-438c-b384-b0e4faca52e5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:08.799780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89db8e-e395-43ed-9c4d-05d9bfa54a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:08.801748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89db8e-e395-43ed-9c4d-05d9bfa54a79 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:08.802956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89db8e-e395-43ed-9c4d-05d9bfa54a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:08.803469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89db8e-e395-43ed-9c4d-05d9bfa54a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:08.804224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89db8e-e395-43ed-9c4d-05d9bfa54a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:08.806285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89db8e-e395-43ed-9c4d-05d9bfa54a79 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:08.952483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1146/2301] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:08 2022] POST /identity/v3/projects => generated 367 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:08.954669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:08.969836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b778ce-7046-458e-87d1-aec0cc129e8f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:09.131984 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7dc95-400b-4ab2-9a2f-ec37e710020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:09.135140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7dc95-400b-4ab2-9a2f-ec37e710020d admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:09.135579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7dc95-400b-4ab2-9a2f-ec37e710020d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:09.136023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7dc95-400b-4ab2-9a2f-ec37e710020d admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:09.137060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7dc95-400b-4ab2-9a2f-ec37e710020d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:09.140761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7dc95-400b-4ab2-9a2f-ec37e710020d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:09.183794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:09.289132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1156/2302] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:08 2022] POST /identity/v3/users => generated 284 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:09.293864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89db8e-e395-43ed-9c4d-05d9bfa54a79 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:09.303945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:09.309890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89db8e-e395-43ed-9c4d-05d9bfa54a79 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:09.328957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9fe66-02af-4ff4-8f5e-c92dcdd00fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:09.329626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fe66-02af-4ff4-8f5e-c92dcdd00fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:09.329914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fe66-02af-4ff4-8f5e-c92dcdd00fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:09.330212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9fe66-02af-4ff4-8f5e-c92dcdd00fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:09.331028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9fe66-02af-4ff4-8f5e-c92dcdd00fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:09.332800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9fe66-02af-4ff4-8f5e-c92dcdd00fc4 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:09.369005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7dc95-400b-4ab2-9a2f-ec37e710020d admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:09.369689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1147/2303] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:09.383932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7dc95-400b-4ab2-9a2f-ec37e710020d admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:09.401620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:09.402282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:09.402565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:09.402836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:09.403500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:09.405236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:09.429438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1157/2304] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:09.445978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9fe66-02af-4ff4-8f5e-c92dcdd00fc4 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:09.460803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9fe66-02af-4ff4-8f5e-c92dcdd00fc4 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:09.483457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090a0fc-a73b-4d81-9093-0f1198fb6e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:09.483457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090a0fc-a73b-4d81-9093-0f1198fb6e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:09.483457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090a0fc-a73b-4d81-9093-0f1198fb6e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:09.483457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090a0fc-a73b-4d81-9093-0f1198fb6e41 admin admin] PATH_INFO: `/v3/projects/64b7f483a40b4510b046abbe5f5f06a7/users/a7f325f254994077a62b744fb4262709/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:09.518410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090a0fc-a73b-4d81-9093-0f1198fb6e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b7f483a40b4510b046abbe5f5f06a7, user_id=a7f325f254994077a62b744fb4262709, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:09.524375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090a0fc-a73b-4d81-9093-0f1198fb6e41 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:09.628164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:09.630970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:09.646688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1148/2305] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:09 2022] PUT /identity/v3/projects/64b7f483a40b4510b046abbe5f5f06a7/users/a7f325f254994077a62b744fb4262709/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:09.648378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cf4cb-6639-4d0f-9a43-0685b2d28970 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:09.648830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cf4cb-6639-4d0f-9a43-0685b2d28970 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:09.649277 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cf4cb-6639-4d0f-9a43-0685b2d28970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:09.680250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a9eea-059c-465c-bfd3-4dd9d93652bf admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:09.697811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-7f4cf4cb-6639-4d0f-9a43-0685b2d28970 None None] MFA Rules not processed for user `a7f325f254994077a62b744fb4262709`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:09.699630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7f4cf4cb-6639-4d0f-9a43-0685b2d28970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:09.847126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:09.847787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:09.848071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:09.848354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] PATH_INFO: `/v3/projects/fca3016ece994fa0b5146e5ff1da819c` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:09.852359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca3016ece994fa0b5146e5ff1da819c)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:09.856500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:09.925491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1149/2306] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:09.938758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cf4cb-6639-4d0f-9a43-0685b2d28970 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:09.941579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7f4cf4cb-6639-4d0f-9a43-0685b2d28970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:10.065600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca3016ece994fa0b5146e5ff1da819c', 'request_id': 'req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.083717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca3016ece994fa0b5146e5ff1da819c', 'request_id': 'req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.094410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cf4cb-6639-4d0f-9a43-0685b2d28970 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:10.173013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58ca47-e0eb-4002-83ca-aa483bb82bf3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c07c85b63514d10bb17b836c60f2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:10.173705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58ca47-e0eb-4002-83ca-aa483bb82bf3 service glance] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:10.174870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58ca47-e0eb-4002-83ca-aa483bb82bf3 service glance] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:10.175462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58ca47-e0eb-4002-83ca-aa483bb82bf3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:10.178056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca3016ece994fa0b5146e5ff1da819c', 'request_id': 'req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.219197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] The token cache is being invalidate because project fca3016ece994fa0b5146e5ff1da819c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:10.219545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.230990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.230990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1158/2307] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:09 2022] DELETE /identity/v3/projects/fca3016ece994fa0b5146e5ff1da819c => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:10.230990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:10.231921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:10.295647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58ca47-e0eb-4002-83ca-aa483bb82bf3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:10.297474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58ca47-e0eb-4002-83ca-aa483bb82bf3 service glance] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:10.309198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1150/2308] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:10.388147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:10.389259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cd519-a05b-4a30-a5f1-72e6b8c7caf7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:10.405631 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:10.406540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:10.406918 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:10.407489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] PATH_INFO: `/v3/users/2be4382bf94347ecba98e563decad26e` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:10.409107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be4382bf94347ecba98e563decad26e)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:10.430110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:10.521615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:10.765123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be4382bf94347ecba98e563decad26e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.893341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be4382bf94347ecba98e563decad26e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.894345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be4382bf94347ecba98e563decad26e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.943486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be4382bf94347ecba98e563decad26e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.960233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-0ab095a8-e126-4efc-8465-eb97fed7bd6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be4382bf94347ecba98e563decad26e'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:10.976424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1159/2309] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:10 2022] DELETE /identity/v3/users/2be4382bf94347ecba98e563decad26e => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:11.270079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:12.578534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:13.965468 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58ca47-e0eb-4002-83ca-aa483bb82bf3 service glance] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:13.993698 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58ca47-e0eb-4002-83ca-aa483bb82bf3 service glance] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:14.364844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:14.369557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:14.369837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:14.371681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] PATH_INFO: `/v3/projects/7e9a8f50400c4ce3bf22c4fcd1c421b0` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:14.381911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9a8f50400c4ce3bf22c4fcd1c421b0)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:14.397415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:14.581867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9a8f50400c4ce3bf22c4fcd1c421b0', 'request_id': 'req-96237419-89a7-48bb-b877-2544539b6ba6'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:14.611770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9a8f50400c4ce3bf22c4fcd1c421b0', 'request_id': 'req-96237419-89a7-48bb-b877-2544539b6ba6'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:14.691332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9a8f50400c4ce3bf22c4fcd1c421b0', 'request_id': 'req-96237419-89a7-48bb-b877-2544539b6ba6'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:14.710932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] The token cache is being invalidate because project 7e9a8f50400c4ce3bf22c4fcd1c421b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:14.711457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:14.712530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-96237419-89a7-48bb-b877-2544539b6ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:14.721508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1151/2310] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:13 2022] DELETE /identity/v3/projects/7e9a8f50400c4ce3bf22c4fcd1c421b0 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:14.728431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c798c8-dcbf-402a-bd27-69975e25f12c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:14.728431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c798c8-dcbf-402a-bd27-69975e25f12c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:14.728431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c798c8-dcbf-402a-bd27-69975e25f12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:14.809067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-15c798c8-dcbf-402a-bd27-69975e25f12c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:14.809995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-15c798c8-dcbf-402a-bd27-69975e25f12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:14.838663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd5d1c-2408-4a3b-9194-5000d25a2298 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:14.839071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd5d1c-2408-4a3b-9194-5000d25a2298 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:14.839447 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd5d1c-2408-4a3b-9194-5000d25a2298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:14.905067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e9cd5d1c-2408-4a3b-9194-5000d25a2298 None None] MFA Rules not processed for user `772205d93ce047e9ad4ddefadec300f4`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:14.906632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e9cd5d1c-2408-4a3b-9194-5000d25a2298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:14.930382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1160/2311] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:14.936408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b37344-4251-413a-a8ea-c0e2757ea159 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:14.936824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b37344-4251-413a-a8ea-c0e2757ea159 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:14.937209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b37344-4251-413a-a8ea-c0e2757ea159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:14.990701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e8b37344-4251-413a-a8ea-c0e2757ea159 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:14.992350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e8b37344-4251-413a-a8ea-c0e2757ea159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:15.006398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1161/2312] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:15.013476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b37344-4251-413a-a8ea-c0e2757ea159 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.030185 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b37344-4251-413a-a8ea-c0e2757ea159 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.039698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.040388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.040670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.040943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.041630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:15.042246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.043807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.048655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:15.061057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1152/2313] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:15.063130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1162/2314] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:56:15.069663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.071470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd5d1c-2408-4a3b-9194-5000d25a2298 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.082747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1b157-825c-422c-b3b8-f3aaa5931615 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.085532 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e9cd5d1c-2408-4a3b-9194-5000d25a2298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:15.091739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42f39d-1fa7-4710-80fe-e57abbc48261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.092829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42f39d-1fa7-4710-80fe-e57abbc48261 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.093304 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42f39d-1fa7-4710-80fe-e57abbc48261 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.093745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42f39d-1fa7-4710-80fe-e57abbc48261 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.094591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42f39d-1fa7-4710-80fe-e57abbc48261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.098065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42f39d-1fa7-4710-80fe-e57abbc48261 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.110263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd5d1c-2408-4a3b-9194-5000d25a2298 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.118926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de005ed-f871-4012-b9b8-322c64da68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.119999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de005ed-f871-4012-b9b8-322c64da68ba admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.120465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de005ed-f871-4012-b9b8-322c64da68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.120740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de005ed-f871-4012-b9b8-322c64da68ba admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.121563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de005ed-f871-4012-b9b8-322c64da68ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.124223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de005ed-f871-4012-b9b8-322c64da68ba admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.213135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1163/2315] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:15 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:15.213135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42f39d-1fa7-4710-80fe-e57abbc48261 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.288442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42f39d-1fa7-4710-80fe-e57abbc48261 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.294837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91802e8c-9626-4bfc-9d29-62ba06aaf575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.295469 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802e8c-9626-4bfc-9d29-62ba06aaf575 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.295747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802e8c-9626-4bfc-9d29-62ba06aaf575 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.296023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802e8c-9626-4bfc-9d29-62ba06aaf575 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.296666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91802e8c-9626-4bfc-9d29-62ba06aaf575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.298800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91802e8c-9626-4bfc-9d29-62ba06aaf575 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.316955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1153/2316] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:15 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:15.316955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de005ed-f871-4012-b9b8-322c64da68ba admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.419032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de005ed-f871-4012-b9b8-322c64da68ba admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.424513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b70eb-368b-4913-a752-c19a78005f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.424772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:15.425185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b70eb-368b-4913-a752-c19a78005f23 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.425465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b70eb-368b-4913-a752-c19a78005f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.425772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b70eb-368b-4913-a752-c19a78005f23 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.426447 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b70eb-368b-4913-a752-c19a78005f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.431588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b70eb-368b-4913-a752-c19a78005f23 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.436115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1164/2317] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:15 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:56:15.440082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91802e8c-9626-4bfc-9d29-62ba06aaf575 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.453674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91802e8c-9626-4bfc-9d29-62ba06aaf575 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.461145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77da9e-f8a8-46c4-97c1-1decb8b6cdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.461145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77da9e-f8a8-46c4-97c1-1decb8b6cdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.461145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77da9e-f8a8-46c4-97c1-1decb8b6cdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.461145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77da9e-f8a8-46c4-97c1-1decb8b6cdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.461738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77da9e-f8a8-46c4-97c1-1decb8b6cdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.463401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77da9e-f8a8-46c4-97c1-1decb8b6cdc6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.475031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1165/2318] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:56:15.479168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77da9e-f8a8-46c4-97c1-1decb8b6cdc6 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.500188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77da9e-f8a8-46c4-97c1-1decb8b6cdc6 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.508736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b665a-a78b-4dec-8fa8-5b9c94aefad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.509843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b665a-a78b-4dec-8fa8-5b9c94aefad0 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.510261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b665a-a78b-4dec-8fa8-5b9c94aefad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.510645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b665a-a78b-4dec-8fa8-5b9c94aefad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.513030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b665a-a78b-4dec-8fa8-5b9c94aefad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.515118 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b665a-a78b-4dec-8fa8-5b9c94aefad0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.525301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1154/2319] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:15 2022] POST /identity/v3/users => generated 295 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:15.530240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b70eb-368b-4913-a752-c19a78005f23 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.536637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1166/2320] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:56:15.538361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b665a-a78b-4dec-8fa8-5b9c94aefad0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.541103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b70eb-368b-4913-a752-c19a78005f23 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.547433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600f197-9eff-4ba9-a0cf-63525ea65b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.548401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600f197-9eff-4ba9-a0cf-63525ea65b13 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.548876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600f197-9eff-4ba9-a0cf-63525ea65b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.549324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600f197-9eff-4ba9-a0cf-63525ea65b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.550000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600f197-9eff-4ba9-a0cf-63525ea65b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.551679 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600f197-9eff-4ba9-a0cf-63525ea65b13 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.556689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b665a-a78b-4dec-8fa8-5b9c94aefad0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.568416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425eac58-303f-4493-81ef-eea64debbb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.569116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eac58-303f-4493-81ef-eea64debbb76 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.570029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eac58-303f-4493-81ef-eea64debbb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.570339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eac58-303f-4493-81ef-eea64debbb76 admin admin] PATH_INFO: `/v3/projects/eaab5df315f34630a055c0700c89bc29/users/7ee21b0824aa4d4b979d8ad15f5c04b7/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.578329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1155/2321] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:15.583920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600f197-9eff-4ba9-a0cf-63525ea65b13 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.604516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425eac58-303f-4493-81ef-eea64debbb76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaab5df315f34630a055c0700c89bc29, user_id=7ee21b0824aa4d4b979d8ad15f5c04b7, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.608275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425eac58-303f-4493-81ef-eea64debbb76 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.625335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600f197-9eff-4ba9-a0cf-63525ea65b13 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.632355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee380d-c626-4861-a4ee-b2d964319987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.633036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee380d-c626-4861-a4ee-b2d964319987 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.633348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee380d-c626-4861-a4ee-b2d964319987 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.633594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee380d-c626-4861-a4ee-b2d964319987 admin admin] PATH_INFO: `/v3/projects/258b970d576d4469bf8592de0f6eaf57/users/e41a2cf9106b444ea98ec068f79b40db/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.669268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee380d-c626-4861-a4ee-b2d964319987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=258b970d576d4469bf8592de0f6eaf57, user_id=e41a2cf9106b444ea98ec068f79b40db, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.672119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee380d-c626-4861-a4ee-b2d964319987 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.794413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1167/2322] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:15 2022] PUT /identity/v3/projects/eaab5df315f34630a055c0700c89bc29/users/7ee21b0824aa4d4b979d8ad15f5c04b7/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:56:15.804215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397734f2-332d-42c5-9da9-1c3e9e0a09a5 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.804501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397734f2-332d-42c5-9da9-1c3e9e0a09a5 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.804776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397734f2-332d-42c5-9da9-1c3e9e0a09a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.855627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1156/2323] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:15 2022] PUT /identity/v3/projects/258b970d576d4469bf8592de0f6eaf57/users/e41a2cf9106b444ea98ec068f79b40db/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:56:15.857052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-397734f2-332d-42c5-9da9-1c3e9e0a09a5 None None] MFA Rules not processed for user `7ee21b0824aa4d4b979d8ad15f5c04b7`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:15.857407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee380d-c626-4861-a4ee-b2d964319987 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:15.859065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-397734f2-332d-42c5-9da9-1c3e9e0a09a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:16.030948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee380d-c626-4861-a4ee-b2d964319987 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.051200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2767335-42e4-4c35-963f-eac478c7bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.053212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2767335-42e4-4c35-963f-eac478c7bb56 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.053340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1168/2324] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:16.053770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2767335-42e4-4c35-963f-eac478c7bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.054053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2767335-42e4-4c35-963f-eac478c7bb56 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.054833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2767335-42e4-4c35-963f-eac478c7bb56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.056605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2767335-42e4-4c35-963f-eac478c7bb56 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.057672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397734f2-332d-42c5-9da9-1c3e9e0a09a5 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:16.077957 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397734f2-332d-42c5-9da9-1c3e9e0a09a5 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.086321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1157/2325] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:16.087819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2767335-42e4-4c35-963f-eac478c7bb56 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:16.089549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03236bee-cc56-48be-9e7b-0a6bf2973786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.090388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03236bee-cc56-48be-9e7b-0a6bf2973786 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.090679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03236bee-cc56-48be-9e7b-0a6bf2973786 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.090956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03236bee-cc56-48be-9e7b-0a6bf2973786 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.091874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03236bee-cc56-48be-9e7b-0a6bf2973786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.094028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03236bee-cc56-48be-9e7b-0a6bf2973786 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.115032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2767335-42e4-4c35-963f-eac478c7bb56 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.120607 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aae3ac-33fe-4a23-b959-e3114c29766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.121528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aae3ac-33fe-4a23-b959-e3114c29766b admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.121862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aae3ac-33fe-4a23-b959-e3114c29766b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.122168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aae3ac-33fe-4a23-b959-e3114c29766b admin admin] PATH_INFO: `/v3/projects/258b970d576d4469bf8592de0f6eaf57/users/e41a2cf9106b444ea98ec068f79b40db/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.124704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aae3ac-33fe-4a23-b959-e3114c29766b admin admin] RBAC: Authorizing `identity:create_grant(project_id=258b970d576d4469bf8592de0f6eaf57, user_id=e41a2cf9106b444ea98ec068f79b40db, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.128696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aae3ac-33fe-4a23-b959-e3114c29766b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.141732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1158/2326] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:16 2022] PUT /identity/v3/projects/258b970d576d4469bf8592de0f6eaf57/users/e41a2cf9106b444ea98ec068f79b40db/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:16.248700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1169/2327] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:16 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:16.279780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682caf76-15fe-4046-9aee-2ff0f2c02800 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.280274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682caf76-15fe-4046-9aee-2ff0f2c02800 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.287120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682caf76-15fe-4046-9aee-2ff0f2c02800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.297652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03236bee-cc56-48be-9e7b-0a6bf2973786 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:16.349885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-682caf76-15fe-4046-9aee-2ff0f2c02800 None None] MFA Rules not processed for user `f2026170f4674b63a37534c14689bd10`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:16.352152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-682caf76-15fe-4046-9aee-2ff0f2c02800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:16.404453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03236bee-cc56-48be-9e7b-0a6bf2973786 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.414472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e5e89-eb37-4c12-9737-6673a6d375ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.415297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e5e89-eb37-4c12-9737-6673a6d375ef admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.415649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e5e89-eb37-4c12-9737-6673a6d375ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.416028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e5e89-eb37-4c12-9737-6673a6d375ef admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.417806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e5e89-eb37-4c12-9737-6673a6d375ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.420967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e5e89-eb37-4c12-9737-6673a6d375ef admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.476683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1159/2328] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:16.481460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682caf76-15fe-4046-9aee-2ff0f2c02800 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:16.483676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-682caf76-15fe-4046-9aee-2ff0f2c02800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:16.506458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682caf76-15fe-4046-9aee-2ff0f2c02800 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.513305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663018ce-d41e-4934-9a3b-ae9cf1d8006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.513936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663018ce-d41e-4934-9a3b-ae9cf1d8006c admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.514267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663018ce-d41e-4934-9a3b-ae9cf1d8006c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.514530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663018ce-d41e-4934-9a3b-ae9cf1d8006c admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.515197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663018ce-d41e-4934-9a3b-ae9cf1d8006c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.517410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663018ce-d41e-4934-9a3b-ae9cf1d8006c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.606825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1170/2329] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:16 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:16.610730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e5e89-eb37-4c12-9737-6673a6d375ef admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:16.626419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e5e89-eb37-4c12-9737-6673a6d375ef admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.644504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a804af8-7320-4b39-bf95-c995d15ce9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.644692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804af8-7320-4b39-bf95-c995d15ce9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.644907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804af8-7320-4b39-bf95-c995d15ce9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.645120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804af8-7320-4b39-bf95-c995d15ce9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.645850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a804af8-7320-4b39-bf95-c995d15ce9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.652097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a804af8-7320-4b39-bf95-c995d15ce9f5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.668388 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1171/2330] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:16.671953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1160/2331] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:16 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:56:16.675213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a804af8-7320-4b39-bf95-c995d15ce9f5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:16.676589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663018ce-d41e-4934-9a3b-ae9cf1d8006c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:16.710660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a804af8-7320-4b39-bf95-c995d15ce9f5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.816360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663018ce-d41e-4934-9a3b-ae9cf1d8006c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.821822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbd3a-6b0a-49fa-86bd-ff4d16cfc6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.822491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbd3a-6b0a-49fa-86bd-ff4d16cfc6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.822826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbd3a-6b0a-49fa-86bd-ff4d16cfc6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.823104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbd3a-6b0a-49fa-86bd-ff4d16cfc6c3 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.823937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bbd3a-6b0a-49fa-86bd-ff4d16cfc6c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.825931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bbd3a-6b0a-49fa-86bd-ff4d16cfc6c3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.842788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0dbec-d67c-48f8-aef6-8ddf7ab66535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.843415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0dbec-d67c-48f8-aef6-8ddf7ab66535 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.843705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0dbec-d67c-48f8-aef6-8ddf7ab66535 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.844123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0dbec-d67c-48f8-aef6-8ddf7ab66535 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.844914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab0dbec-d67c-48f8-aef6-8ddf7ab66535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.847040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab0dbec-d67c-48f8-aef6-8ddf7ab66535 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.881402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1172/2332] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:16.883067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0dbec-d67c-48f8-aef6-8ddf7ab66535 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:16.914870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0dbec-d67c-48f8-aef6-8ddf7ab66535 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:16.924714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3a8d-0313-4785-b9b7-e08b16e4ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:16.926275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3a8d-0313-4785-b9b7-e08b16e4ff30 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:16.927781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3a8d-0313-4785-b9b7-e08b16e4ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:16.928487 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3a8d-0313-4785-b9b7-e08b16e4ff30 admin admin] PATH_INFO: `/v3/projects/5a2fc40c31104055ba21666486c5f1bc/users/a00a390c60a94963841c1f6b5f049d4d/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:16.960013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3a8d-0313-4785-b9b7-e08b16e4ff30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2fc40c31104055ba21666486c5f1bc, user_id=a00a390c60a94963841c1f6b5f049d4d, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:16.966408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3a8d-0313-4785-b9b7-e08b16e4ff30 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:16.986188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1161/2333] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:16 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:16.992167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbd3a-6b0a-49fa-86bd-ff4d16cfc6c3 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:17.024412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbd3a-6b0a-49fa-86bd-ff4d16cfc6c3 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:17.051204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfd71a-34e6-481c-a68b-6c953777c20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:17.052070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd71a-34e6-481c-a68b-6c953777c20b admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:17.053201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd71a-34e6-481c-a68b-6c953777c20b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:17.054463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd71a-34e6-481c-a68b-6c953777c20b admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:17.056922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfd71a-34e6-481c-a68b-6c953777c20b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:17.059882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfd71a-34e6-481c-a68b-6c953777c20b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:17.091588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1162/2334] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:56:17.092746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfd71a-34e6-481c-a68b-6c953777c20b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:17.105272 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfd71a-34e6-481c-a68b-6c953777c20b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:17.117928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1173/2335] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:16 2022] PUT /identity/v3/projects/5a2fc40c31104055ba21666486c5f1bc/users/a00a390c60a94963841c1f6b5f049d4d/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:17.144218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d9ff-0aea-4ee9-aea3-1602335e3850 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:17.144503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d9ff-0aea-4ee9-aea3-1602335e3850 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:17.144779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d9ff-0aea-4ee9-aea3-1602335e3850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:17.233211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a0c6d9ff-0aea-4ee9-aea3-1602335e3850 None None] MFA Rules not processed for user `a00a390c60a94963841c1f6b5f049d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:17.234359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a0c6d9ff-0aea-4ee9-aea3-1602335e3850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:17.297022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03f84a-12e8-4420-84ab-b7c33c5ad212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:17.298026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03f84a-12e8-4420-84ab-b7c33c5ad212 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:17.298944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03f84a-12e8-4420-84ab-b7c33c5ad212 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:17.299412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03f84a-12e8-4420-84ab-b7c33c5ad212 admin admin] PATH_INFO: `/v3/projects/3f67c7acd4c9459c9e864526acd53eca/users/b153d726cb7b4193a7cf5516306fa6fd/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:17.334775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03f84a-12e8-4420-84ab-b7c33c5ad212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f67c7acd4c9459c9e864526acd53eca, user_id=b153d726cb7b4193a7cf5516306fa6fd, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:17.337697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03f84a-12e8-4420-84ab-b7c33c5ad212 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:17.439730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1163/2336] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:17 2022] PUT /identity/v3/projects/3f67c7acd4c9459c9e864526acd53eca/users/b153d726cb7b4193a7cf5516306fa6fd/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:17.445215 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03f84a-12e8-4420-84ab-b7c33c5ad212 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:17.486756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1174/2337] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:17.498725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d9ff-0aea-4ee9-aea3-1602335e3850 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:17.547352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d9ff-0aea-4ee9-aea3-1602335e3850 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:17.728244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03f84a-12e8-4420-84ab-b7c33c5ad212 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:17.746678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb282b-3437-4e9b-a5d6-963692c27899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:17.749151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb282b-3437-4e9b-a5d6-963692c27899 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:17.749151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb282b-3437-4e9b-a5d6-963692c27899 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:17.749151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb282b-3437-4e9b-a5d6-963692c27899 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:17.756134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb282b-3437-4e9b-a5d6-963692c27899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:17.757813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb282b-3437-4e9b-a5d6-963692c27899 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:17.767031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b332596-61f4-4bb0-b71b-267b9d92ec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:17.767915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332596-61f4-4bb0-b71b-267b9d92ec1e admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:17.768534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332596-61f4-4bb0-b71b-267b9d92ec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:17.769047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332596-61f4-4bb0-b71b-267b9d92ec1e admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:17.769901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b332596-61f4-4bb0-b71b-267b9d92ec1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:17.772522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b332596-61f4-4bb0-b71b-267b9d92ec1e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:17.772973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1164/2338] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:17.785346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb282b-3437-4e9b-a5d6-963692c27899 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:17.802498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb282b-3437-4e9b-a5d6-963692c27899 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:17.808458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820032-a8c0-4158-bfe6-5301c867ddd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:17.809260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820032-a8c0-4158-bfe6-5301c867ddd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:17.809643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820032-a8c0-4158-bfe6-5301c867ddd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:17.809981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820032-a8c0-4158-bfe6-5301c867ddd1 admin admin] PATH_INFO: `/v3/projects/3f67c7acd4c9459c9e864526acd53eca/users/b153d726cb7b4193a7cf5516306fa6fd/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:17.813076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820032-a8c0-4158-bfe6-5301c867ddd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f67c7acd4c9459c9e864526acd53eca, user_id=b153d726cb7b4193a7cf5516306fa6fd, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:17.816110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820032-a8c0-4158-bfe6-5301c867ddd1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:17.833629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1165/2339] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:17 2022] PUT /identity/v3/projects/3f67c7acd4c9459c9e864526acd53eca/users/b153d726cb7b4193a7cf5516306fa6fd/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:17.938858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1175/2340] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:17 2022] POST /identity/v3/projects => generated 383 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:17.946957 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820032-a8c0-4158-bfe6-5301c867ddd1 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:18.063105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820032-a8c0-4158-bfe6-5301c867ddd1 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:18.161004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba9d93-7d01-4e7c-b9b6-22e240369691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:18.162373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba9d93-7d01-4e7c-b9b6-22e240369691 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.162649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba9d93-7d01-4e7c-b9b6-22e240369691 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.163176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba9d93-7d01-4e7c-b9b6-22e240369691 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.163943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba9d93-7d01-4e7c-b9b6-22e240369691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:18.166216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba9d93-7d01-4e7c-b9b6-22e240369691 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:18.278054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:18.379030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1166/2341] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:17 2022] POST /identity/v3/users => generated 298 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:18.389802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b332596-61f4-4bb0-b71b-267b9d92ec1e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:18.402335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b332596-61f4-4bb0-b71b-267b9d92ec1e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:18.409366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1334d17-e0f5-4c54-a408-339e0faa73c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:18.410607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1334d17-e0f5-4c54-a408-339e0faa73c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.410985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1334d17-e0f5-4c54-a408-339e0faa73c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.411950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1334d17-e0f5-4c54-a408-339e0faa73c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.413727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1334d17-e0f5-4c54-a408-339e0faa73c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:18.422752 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1334d17-e0f5-4c54-a408-339e0faa73c4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:18.463059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1176/2342] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:18.468271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba9d93-7d01-4e7c-b9b6-22e240369691 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:18.492921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba9d93-7d01-4e7c-b9b6-22e240369691 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:18.513350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c5e0e-67fd-4ee2-9fbe-3a9e47f3bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:18.515389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c5e0e-67fd-4ee2-9fbe-3a9e47f3bed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.517060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c5e0e-67fd-4ee2-9fbe-3a9e47f3bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.517627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c5e0e-67fd-4ee2-9fbe-3a9e47f3bed2 admin admin] PATH_INFO: `/v3/projects/26fe78c06b674acc89632d246838b774/users/d0d3d590531d467bb66e08b820d19cc0/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.548730 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c5e0e-67fd-4ee2-9fbe-3a9e47f3bed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26fe78c06b674acc89632d246838b774, user_id=d0d3d590531d467bb66e08b820d19cc0, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:18.551651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c5e0e-67fd-4ee2-9fbe-3a9e47f3bed2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:18.641801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1167/2343] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:18 2022] PUT /identity/v3/projects/26fe78c06b674acc89632d246838b774/users/d0d3d590531d467bb66e08b820d19cc0/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:18.644117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1334d17-e0f5-4c54-a408-339e0faa73c4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:18.658453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1334d17-e0f5-4c54-a408-339e0faa73c4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:18.784586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9cb03-dcf7-44c0-a60f-ee12af4ab791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:18.785224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9cb03-dcf7-44c0-a60f-ee12af4ab791 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.785575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9cb03-dcf7-44c0-a60f-ee12af4ab791 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.785941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9cb03-dcf7-44c0-a60f-ee12af4ab791 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.787356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9cb03-dcf7-44c0-a60f-ee12af4ab791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:18.789950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9cb03-dcf7-44c0-a60f-ee12af4ab791 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:18.824047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1177/2344] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:18.825492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c5e0e-67fd-4ee2-9fbe-3a9e47f3bed2 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:18.841147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c5e0e-67fd-4ee2-9fbe-3a9e47f3bed2 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:18.849958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:18.851183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.852204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.852671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] PATH_INFO: `/v3/projects/26fe78c06b674acc89632d246838b774/users/d0d3d590531d467bb66e08b820d19cc0/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.855518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] RBAC: Authorizing `identity:create_grant(project_id=26fe78c06b674acc89632d246838b774, user_id=d0d3d590531d467bb66e08b820d19cc0, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:18.858846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:18.882097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1168/2345] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:18 2022] PUT /identity/v3/projects/26fe78c06b674acc89632d246838b774/users/d0d3d590531d467bb66e08b820d19cc0/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:18.893176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d44d8-4976-484d-ad7f-d90525b4210b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.893793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d44d8-4976-484d-ad7f-d90525b4210b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.894295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d44d8-4976-484d-ad7f-d90525b4210b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.947920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:18.980482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ab0d44d8-4976-484d-ad7f-d90525b4210b None None] MFA Rules not processed for user `d0d3d590531d467bb66e08b820d19cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:18.984046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ab0d44d8-4976-484d-ad7f-d90525b4210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:19.424638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1178/2346] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 532 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:19.444007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:19.446857 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:19.566659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb754a3-a55e-4a0e-b2fc-8f20935db83f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:19.577559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408c556-f564-42fe-8822-d54ab44e3b1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:19.578684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408c556-f564-42fe-8822-d54ab44e3b1a service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:19.580423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408c556-f564-42fe-8822-d54ab44e3b1a service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:19.581025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408c556-f564-42fe-8822-d54ab44e3b1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:19.797004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408c556-f564-42fe-8822-d54ab44e3b1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:19.797004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408c556-f564-42fe-8822-d54ab44e3b1a service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:19.798267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d44d8-4976-484d-ad7f-d90525b4210b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:19.801204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ab0d44d8-4976-484d-ad7f-d90525b4210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:19.872046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1169/2347] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:19.952043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d44d8-4976-484d-ad7f-d90525b4210b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:19.972870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:19.974564 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:19.975165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:19.976012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] PATH_INFO: `/v3/users/65ed045641eb44389410527d391a22db` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:19.978799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ed045641eb44389410527d391a22db)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:19.982014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:20.326463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ed045641eb44389410527d391a22db'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:20.441758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ed045641eb44389410527d391a22db'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:20.442996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ed045641eb44389410527d391a22db'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:20.473234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ed045641eb44389410527d391a22db'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:20.508989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ed045641eb44389410527d391a22db'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:20.542309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1179/2348] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:19 2022] DELETE /identity/v3/users/65ed045641eb44389410527d391a22db => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:20.546379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe627-f8d5-4c5b-b942-557a3030e8be None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:20.547323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe627-f8d5-4c5b-b942-557a3030e8be None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:20.547323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe627-f8d5-4c5b-b942-557a3030e8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:20.591479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-fedfe627-f8d5-4c5b-b942-557a3030e8be None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:20.595850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fedfe627-f8d5-4c5b-b942-557a3030e8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:20.787524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1170/2349] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:20.803228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:20.805478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:20.983785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c7487-2190-4587-a4c8-6a0b615fa0c7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:20.999607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:21.000785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:21.001251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:21.001717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:21.047604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:21.049208 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:21.087551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1180/2350] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:21.100533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:21.830129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:22.196221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:22.411436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfe627-f8d5-4c5b-b942-557a3030e8be None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:22.413392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fedfe627-f8d5-4c5b-b942-557a3030e8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:22.435493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfe627-f8d5-4c5b-b942-557a3030e8be None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:22.452568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:22.454429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:22.455841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:22.456295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] PATH_INFO: `/v3/projects/1ba8fd6038c742da8fbd814b9d1f02ba` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:22.458219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba8fd6038c742da8fbd814b9d1f02ba)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:22.471113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:22.607127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:22.613260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:22.681983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba8fd6038c742da8fbd814b9d1f02ba', 'request_id': 'req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:22.691801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba8fd6038c742da8fbd814b9d1f02ba', 'request_id': 'req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:22.751909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b9818-5c8f-47ae-9c81-3117d5c0e05f service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:22.827665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf257-3582-449f-8614-2780d2dfb7fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:22.836371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf257-3582-449f-8614-2780d2dfb7fb service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:22.836371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf257-3582-449f-8614-2780d2dfb7fb service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:22.836371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf257-3582-449f-8614-2780d2dfb7fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:22.836660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba8fd6038c742da8fbd814b9d1f02ba', 'request_id': 'req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:22.836768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d10cf257-3582-449f-8614-2780d2dfb7fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:22.860726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] The token cache is being invalidate because project 1ba8fd6038c742da8fbd814b9d1f02ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:22.861034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:22.862302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e0067cdf-3efa-4bfa-9bd0-de5fa3998468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:22.864676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1171/2351] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:22 2022] DELETE /identity/v3/projects/1ba8fd6038c742da8fbd814b9d1f02ba => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:22.881755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1738-35a9-457c-9c92-94ba6f91d784 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:22.882059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1738-35a9-457c-9c92-94ba6f91d784 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:22.882396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1738-35a9-457c-9c92-94ba6f91d784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:22.952340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-69df1738-35a9-457c-9c92-94ba6f91d784 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:22.954797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-69df1738-35a9-457c-9c92-94ba6f91d784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:22.979928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf257-3582-449f-8614-2780d2dfb7fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:22.981872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf257-3582-449f-8614-2780d2dfb7fb service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:23.011442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1181/2352] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:23.167617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1172/2353] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:23.175608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6febad1-4ed4-47a0-9b96-08f2482b5187 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:23.175924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6febad1-4ed4-47a0-9b96-08f2482b5187 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:23.178484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6febad1-4ed4-47a0-9b96-08f2482b5187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:23.236400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e6febad1-4ed4-47a0-9b96-08f2482b5187 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:23.240400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e6febad1-4ed4-47a0-9b96-08f2482b5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:23.289223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df1738-35a9-457c-9c92-94ba6f91d784 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:23.292005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1182/2354] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:23.307362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df1738-35a9-457c-9c92-94ba6f91d784 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:23.320434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:23.321068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:23.321347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:23.325628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:23.328633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:23.330010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:23.335398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:23.362874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1173/2355] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:56:23.367566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6febad1-4ed4-47a0-9b96-08f2482b5187 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:23.381065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6febad1-4ed4-47a0-9b96-08f2482b5187 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:23.399221 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baca14-e5be-40eb-875e-757bce92ae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:23.400127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baca14-e5be-40eb-875e-757bce92ae2d admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:23.400558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baca14-e5be-40eb-875e-757bce92ae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:23.400981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baca14-e5be-40eb-875e-757bce92ae2d admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:23.402109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baca14-e5be-40eb-875e-757bce92ae2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:23.404370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baca14-e5be-40eb-875e-757bce92ae2d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:23.628932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1183/2356] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:23 2022] POST /identity/v3/projects => generated 383 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:23.634752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:23.654844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2711cf-c813-4543-8581-4a397c67ab60 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:23.787361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e14f7a-f090-4fe1-a8a8-e9b05a4338dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:23.788154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14f7a-f090-4fe1-a8a8-e9b05a4338dd admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:23.788613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14f7a-f090-4fe1-a8a8-e9b05a4338dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:23.789231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14f7a-f090-4fe1-a8a8-e9b05a4338dd admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:23.790082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e14f7a-f090-4fe1-a8a8-e9b05a4338dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:23.792665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e14f7a-f090-4fe1-a8a8-e9b05a4338dd admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:23.883128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1174/2357] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:23 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:23.887462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baca14-e5be-40eb-875e-757bce92ae2d admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:23.918911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baca14-e5be-40eb-875e-757bce92ae2d admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:23.930566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db61af-43fd-4b9a-8e80-0f29dc43af9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:23.931289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db61af-43fd-4b9a-8e80-0f29dc43af9a admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:23.931471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db61af-43fd-4b9a-8e80-0f29dc43af9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:23.932502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db61af-43fd-4b9a-8e80-0f29dc43af9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:23.933035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db61af-43fd-4b9a-8e80-0f29dc43af9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:23.935261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db61af-43fd-4b9a-8e80-0f29dc43af9a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:24.005028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e14f7a-f090-4fe1-a8a8-e9b05a4338dd admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:24.010860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1184/2358] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:24.021302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e14f7a-f090-4fe1-a8a8-e9b05a4338dd admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:24.044742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44502efc-6932-4c5e-a134-f989574e198c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:24.045394 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502efc-6932-4c5e-a134-f989574e198c admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:24.046616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502efc-6932-4c5e-a134-f989574e198c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:24.048046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502efc-6932-4c5e-a134-f989574e198c admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:24.050083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44502efc-6932-4c5e-a134-f989574e198c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:24.056770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44502efc-6932-4c5e-a134-f989574e198c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:24.129228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1175/2359] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:56:24.134298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db61af-43fd-4b9a-8e80-0f29dc43af9a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:24.171691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db61af-43fd-4b9a-8e80-0f29dc43af9a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:24.214444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:24.215351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:24.215797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:24.216234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] PATH_INFO: `/v3/projects/1363660dde8f4e4295adc44c6ab87f8b/users/5ae9377cf36e4086aa38c07b021fdead/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:24.268130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1363660dde8f4e4295adc44c6ab87f8b, user_id=5ae9377cf36e4086aa38c07b021fdead, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:24.276280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:24.453652 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1185/2360] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:24 2022] PUT /identity/v3/projects/1363660dde8f4e4295adc44c6ab87f8b/users/5ae9377cf36e4086aa38c07b021fdead/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:24.455061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfefd8-46a1-41b4-b408-0f13acc63047 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:24.455546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfefd8-46a1-41b4-b408-0f13acc63047 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:24.455673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfefd8-46a1-41b4-b408-0f13acc63047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:24.500935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f2bfefd8-46a1-41b4-b408-0f13acc63047 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:24.502269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f2bfefd8-46a1-41b4-b408-0f13acc63047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:24.695196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1176/2361] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:24.714974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:24.717669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:24.846059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6d2f4-0366-474d-a8c6-55e4e07df22f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:24.853315 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed3fbc-7d45-432d-be5e-31bb3e879363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:24.854260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed3fbc-7d45-432d-be5e-31bb3e879363 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:24.854751 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed3fbc-7d45-432d-be5e-31bb3e879363 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:24.855240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed3fbc-7d45-432d-be5e-31bb3e879363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:24.868478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed3fbc-7d45-432d-be5e-31bb3e879363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:24.870281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed3fbc-7d45-432d-be5e-31bb3e879363 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:24.884716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1186/2362] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:25.086340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:25.440739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:25.771807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfefd8-46a1-41b4-b408-0f13acc63047 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:25.774053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f2bfefd8-46a1-41b4-b408-0f13acc63047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:25.873467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfefd8-46a1-41b4-b408-0f13acc63047 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:25.879793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc742dfa-8d15-4b8e-8322-adf8defc4584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:25.883956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc742dfa-8d15-4b8e-8322-adf8defc4584 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:25.886964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc742dfa-8d15-4b8e-8322-adf8defc4584 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:25.886964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc742dfa-8d15-4b8e-8322-adf8defc4584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:25.888644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cc742dfa-8d15-4b8e-8322-adf8defc4584 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:26.028410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc742dfa-8d15-4b8e-8322-adf8defc4584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:26.029887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc742dfa-8d15-4b8e-8322-adf8defc4584 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:26.042797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1177/2363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:27.122428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:27.608469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:28.091474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1460ed-ab89-48d3-9c16-d32022bc99b0 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:28.091768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1460ed-ab89-48d3-9c16-d32022bc99b0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:28.092054 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1460ed-ab89-48d3-9c16-d32022bc99b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:28.128806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5d1460ed-ab89-48d3-9c16-d32022bc99b0 None None] MFA Rules not processed for user `eca89f435299415c8448931e73f5b0fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:28.130647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5d1460ed-ab89-48d3-9c16-d32022bc99b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:28.311845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1187/2364] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:29.260002 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:29.569873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640d99-33c4-424d-b0ae-c68cbb2be540 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:29.570922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640d99-33c4-424d-b0ae-c68cbb2be540 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:29.571731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640d99-33c4-424d-b0ae-c68cbb2be540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:29.605415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:29.653515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e4640d99-33c4-424d-b0ae-c68cbb2be540 None None] MFA Rules not processed for user `5ae9377cf36e4086aa38c07b021fdead`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:29.654840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e4640d99-33c4-424d-b0ae-c68cbb2be540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:29.876382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1178/2365] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:29.889067 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1460ed-ab89-48d3-9c16-d32022bc99b0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:29.890969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5d1460ed-ab89-48d3-9c16-d32022bc99b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:29.916767 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1460ed-ab89-48d3-9c16-d32022bc99b0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:29.926977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:29.928384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:29.928837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:29.929259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:29.949531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:29.952762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:29.976135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1188/2366] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:30.713677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93fa2-2cad-4d65-a60e-aa30986b9006 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:30.714754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93fa2-2cad-4d65-a60e-aa30986b9006 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:30.722782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93fa2-2cad-4d65-a60e-aa30986b9006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:30.781222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c9c93fa2-2cad-4d65-a60e-aa30986b9006 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:30.810308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c9c93fa2-2cad-4d65-a60e-aa30986b9006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:30.992331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1179/2367] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:56:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:31.065444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:31.070035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:31.349845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dbb37-552c-44d7-911d-dd2f715bdfc1 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:31.357049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:31.357994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:31.365300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:31.365300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:31.371993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:31.373810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:31.405777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1189/2368] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 13:56:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:56:31.418375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:32.073312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:32.400802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3d8f-1bba-45a7-8037-4f2ca3b8def2 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:32.401084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3d8f-1bba-45a7-8037-4f2ca3b8def2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:32.401703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3d8f-1bba-45a7-8037-4f2ca3b8def2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:32.506316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c64e3d8f-1bba-45a7-8037-4f2ca3b8def2 None None] MFA Rules not processed for user `e41a2cf9106b444ea98ec068f79b40db`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:32.507875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c64e3d8f-1bba-45a7-8037-4f2ca3b8def2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:32.766631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1180/2369] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:32.789131 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:32.792053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:32.997617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd56057-fd49-44fb-aed4-8b0181a5f5cf service swift] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:33.013196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:33.017727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:33.019437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:33.020012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:33.029249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:33.219720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:33.222102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:33.262003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1190/2370] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:33.918161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48b7e-7dad-4db3-a5cb-f9430e78d452 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:33.918659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48b7e-7dad-4db3-a5cb-f9430e78d452 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:33.918659 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48b7e-7dad-4db3-a5cb-f9430e78d452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:33.982850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c4c48b7e-7dad-4db3-a5cb-f9430e78d452 None None] MFA Rules not processed for user `5ae9377cf36e4086aa38c07b021fdead`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:33.984388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c4c48b7e-7dad-4db3-a5cb-f9430e78d452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:34.025395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1181/2371] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:34.067100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:34.075381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:34.095949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2c7f9-9ef9-4b03-9515-e35840c01d4e service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:34.121700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:34.124703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:34.128680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:34.131856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] PATH_INFO: `/v3/users/7ee21b0824aa4d4b979d8ad15f5c04b7` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:34.139170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee21b0824aa4d4b979d8ad15f5c04b7)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:34.142095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:34.421524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141731ef-92bf-4925-a8e7-cbb080e618eb None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:34.421840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141731ef-92bf-4925-a8e7-cbb080e618eb None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:34.422118 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141731ef-92bf-4925-a8e7-cbb080e618eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:34.478849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-141731ef-92bf-4925-a8e7-cbb080e618eb None None] MFA Rules not processed for user `b153d726cb7b4193a7cf5516306fa6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:34.481259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-141731ef-92bf-4925-a8e7-cbb080e618eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:34.517249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee21b0824aa4d4b979d8ad15f5c04b7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:34.620733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee21b0824aa4d4b979d8ad15f5c04b7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:34.623803 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee21b0824aa4d4b979d8ad15f5c04b7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:34.675308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee21b0824aa4d4b979d8ad15f5c04b7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:34.691407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-b71eeadd-22db-4f3f-98cf-04c909cd1541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee21b0824aa4d4b979d8ad15f5c04b7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:34.704991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1191/2372] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:34 2022] DELETE /identity/v3/users/7ee21b0824aa4d4b979d8ad15f5c04b7 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:34.710857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed010407-fe70-4a4b-8f17-621e5e361678 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:34.711160 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed010407-fe70-4a4b-8f17-621e5e361678 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:34.711451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed010407-fe70-4a4b-8f17-621e5e361678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:34.743055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1182/2373] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:34.772106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-ed010407-fe70-4a4b-8f17-621e5e361678 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:34.773653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ed010407-fe70-4a4b-8f17-621e5e361678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:34.780400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141731ef-92bf-4925-a8e7-cbb080e618eb None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:34.784384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-141731ef-92bf-4925-a8e7-cbb080e618eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:34.813925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141731ef-92bf-4925-a8e7-cbb080e618eb None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:34.948049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1192/2374] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:34.956212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed010407-fe70-4a4b-8f17-621e5e361678 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:34.957681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ed010407-fe70-4a4b-8f17-621e5e361678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:34.981262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:34.984136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:34.984456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:34.984740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:34.989022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:35.088055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed010407-fe70-4a4b-8f17-621e5e361678 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:35.099073 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:35.100228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:35.100511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:35.100974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:35.121247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:35.123128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:35.153204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:35.153717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1193/2375] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:56:35.155772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:35.182009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1183/2376] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:35.390791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:36.234883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:36.237955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:36.299587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040716-efb8-4afa-b248-0471a97b8543 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:36.321001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:36.321001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:36.321001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:36.321001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] PATH_INFO: `/v3/projects/eaab5df315f34630a055c0700c89bc29` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:36.323777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaab5df315f34630a055c0700c89bc29)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:36.330753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:36.661210 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:36.688648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaab5df315f34630a055c0700c89bc29', 'request_id': 'req-4582820c-1ff1-4f36-bbb7-02a0d3a01431'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:36.702109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaab5df315f34630a055c0700c89bc29', 'request_id': 'req-4582820c-1ff1-4f36-bbb7-02a0d3a01431'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:36.893479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaab5df315f34630a055c0700c89bc29', 'request_id': 'req-4582820c-1ff1-4f36-bbb7-02a0d3a01431'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:36.926294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] The token cache is being invalidate because project eaab5df315f34630a055c0700c89bc29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:36.926680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:36.937258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:36.947810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1194/2377] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:36 2022] DELETE /identity/v3/projects/eaab5df315f34630a055c0700c89bc29 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:36.954229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:36.996798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3681283-15a2-48ed-9ec9-1f12ef04b9b1 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:37.191541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:37.192502 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:37.193497 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:37.195899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] PATH_INFO: `/v3/users/a00a390c60a94963841c1f6b5f049d4d` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:37.204135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a00a390c60a94963841c1f6b5f049d4d)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:37.230890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:37.666106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a00a390c60a94963841c1f6b5f049d4d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:37.756058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a00a390c60a94963841c1f6b5f049d4d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:37.757600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a00a390c60a94963841c1f6b5f049d4d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:37.758587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a00a390c60a94963841c1f6b5f049d4d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:37.771327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a00a390c60a94963841c1f6b5f049d4d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:37.830554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1184/2378] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:36 2022] DELETE /identity/v3/users/a00a390c60a94963841c1f6b5f049d4d => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:38.770220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:39.285706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:39.293605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:39.353446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582820c-1ff1-4f36-bbb7-02a0d3a01431 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:39.544484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:39.545566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:39.546041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:39.546551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] PATH_INFO: `/v3/projects/5a2fc40c31104055ba21666486c5f1bc` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:39.548619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a2fc40c31104055ba21666486c5f1bc)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:39.552120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:39.848585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a2fc40c31104055ba21666486c5f1bc', 'request_id': 'req-c196c1b0-45f4-4da1-a76c-abcccadebc28'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:39.867133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a2fc40c31104055ba21666486c5f1bc', 'request_id': 'req-c196c1b0-45f4-4da1-a76c-abcccadebc28'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.017027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a2fc40c31104055ba21666486c5f1bc', 'request_id': 'req-c196c1b0-45f4-4da1-a76c-abcccadebc28'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.060556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] The token cache is being invalidate because project 5a2fc40c31104055ba21666486c5f1bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:40.061367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.062490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.065963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1195/2379] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:39 2022] DELETE /identity/v3/projects/5a2fc40c31104055ba21666486c5f1bc => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:40.070514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:40.073309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:40.100838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821a505-209b-4cd9-b766-1104a9e059a2 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:40.215454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:40.297789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:40.303856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:40.304143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:40.304425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] PATH_INFO: `/v3/users/d0d3d590531d467bb66e08b820d19cc0` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:40.307775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d3d590531d467bb66e08b820d19cc0)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:40.309806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:40.534990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:40.682445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d3d590531d467bb66e08b820d19cc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.812621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d3d590531d467bb66e08b820d19cc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.813694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d3d590531d467bb66e08b820d19cc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.814626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d3d590531d467bb66e08b820d19cc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.850185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ef5e6997-8b57-4dc9-bf13-ca4b3bdce9a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d3d590531d467bb66e08b820d19cc0'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:40.875552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1185/2380] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:40 2022] DELETE /identity/v3/users/d0d3d590531d467bb66e08b820d19cc0 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:41.514527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:41.779748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:41.782529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:41.816103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196c1b0-45f4-4da1-a76c-abcccadebc28 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:42.034317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:42.034997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:42.035281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:42.037455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] PATH_INFO: `/v3/projects/26fe78c06b674acc89632d246838b774` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:42.041772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26fe78c06b674acc89632d246838b774)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:42.043792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:42.252268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26fe78c06b674acc89632d246838b774', 'request_id': 'req-ecd28ac0-3033-4778-b51d-53a16bb0e930'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:42.273834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26fe78c06b674acc89632d246838b774', 'request_id': 'req-ecd28ac0-3033-4778-b51d-53a16bb0e930'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:42.352261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26fe78c06b674acc89632d246838b774', 'request_id': 'req-ecd28ac0-3033-4778-b51d-53a16bb0e930'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:42.372713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] The token cache is being invalidate because project 26fe78c06b674acc89632d246838b774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:42.374022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:42.375289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecd28ac0-3033-4778-b51d-53a16bb0e930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:42.381083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1196/2381] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:41 2022] DELETE /identity/v3/projects/26fe78c06b674acc89632d246838b774 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:42.382457 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6490f48-bfb4-42c0-ad32-c345764d0192 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:42.382754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6490f48-bfb4-42c0-ad32-c345764d0192 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:42.383031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6490f48-bfb4-42c0-ad32-c345764d0192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:42.409393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b6490f48-bfb4-42c0-ad32-c345764d0192 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:42.411331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b6490f48-bfb4-42c0-ad32-c345764d0192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:42.573368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1186/2382] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:42.583325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701e2c3-2154-4ae5-a870-cbd355b2d534 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:42.584608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701e2c3-2154-4ae5-a870-cbd355b2d534 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:42.584954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701e2c3-2154-4ae5-a870-cbd355b2d534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:42.630356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-9701e2c3-2154-4ae5-a870-cbd355b2d534 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:42.630356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9701e2c3-2154-4ae5-a870-cbd355b2d534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:42.631076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1197/2383] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:42.639305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6490f48-bfb4-42c0-ad32-c345764d0192 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:42.651523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:42.655628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6490f48-bfb4-42c0-ad32-c345764d0192 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:42.663819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:42.664462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:42.664748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:42.665466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:42.665998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:42.666496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:42.668532 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:42.684267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1187/2384] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:56:42.689481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701e2c3-2154-4ae5-a870-cbd355b2d534 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:42.700184 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701e2c3-2154-4ae5-a870-cbd355b2d534 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:42.713111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5489e1-8d27-4b94-a3ec-fce1a337642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:42.714017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5489e1-8d27-4b94-a3ec-fce1a337642c admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:42.714908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5489e1-8d27-4b94-a3ec-fce1a337642c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:42.715347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5489e1-8d27-4b94-a3ec-fce1a337642c admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:42.716215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5489e1-8d27-4b94-a3ec-fce1a337642c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:42.719149 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5489e1-8d27-4b94-a3ec-fce1a337642c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:42.941629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:42.943781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1198/2385] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:42 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:56:42.965360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fe29b-dc3f-41eb-85fc-166d30ebaedb admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:43.120281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ce04-c1c5-4252-894f-673672cf2f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:43.120949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ce04-c1c5-4252-894f-673672cf2f7f admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:43.121236 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ce04-c1c5-4252-894f-673672cf2f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:43.121564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ce04-c1c5-4252-894f-673672cf2f7f admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:43.122766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2ce04-c1c5-4252-894f-673672cf2f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:43.126569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2ce04-c1c5-4252-894f-673672cf2f7f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:43.147497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ce68-957d-4065-b839-4fb3273a2cab None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:43.147863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ce68-957d-4065-b839-4fb3273a2cab None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:43.148152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302ce68-957d-4065-b839-4fb3273a2cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:43.167601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5302ce68-957d-4065-b839-4fb3273a2cab None None] MFA Rules not processed for user `f2026170f4674b63a37534c14689bd10`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:43.168839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5302ce68-957d-4065-b839-4fb3273a2cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:43.234252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1199/2386] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:43.252920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1188/2387] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:42 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:56:43.258782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302ce68-957d-4065-b839-4fb3273a2cab None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:43.268318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302ce68-957d-4065-b839-4fb3273a2cab None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:43.274126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af59f0-6c90-49ed-830c-c0586852a8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:43.274954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af59f0-6c90-49ed-830c-c0586852a8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:43.275242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af59f0-6c90-49ed-830c-c0586852a8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:43.275521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af59f0-6c90-49ed-830c-c0586852a8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:43.276833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af59f0-6c90-49ed-830c-c0586852a8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:43.277960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af59f0-6c90-49ed-830c-c0586852a8f6 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:43.299866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1200/2388] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:43.300975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ce04-c1c5-4252-894f-673672cf2f7f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:43.313654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ce04-c1c5-4252-894f-673672cf2f7f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:43.321276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426a5a0-a4c6-419e-abcb-b70c9a521770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:43.322020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426a5a0-a4c6-419e-abcb-b70c9a521770 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:43.323108 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426a5a0-a4c6-419e-abcb-b70c9a521770 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:43.323398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426a5a0-a4c6-419e-abcb-b70c9a521770 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:43.324150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426a5a0-a4c6-419e-abcb-b70c9a521770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:43.325851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426a5a0-a4c6-419e-abcb-b70c9a521770 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:43.338104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1189/2389] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:43.343087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af59f0-6c90-49ed-830c-c0586852a8f6 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:43.355697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af59f0-6c90-49ed-830c-c0586852a8f6 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:43.363968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:43.365436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:43.365860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:43.366271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] PATH_INFO: `/v3/projects/f8e8da336a5f4a5c8d5d4ea5bfcfa639/users/076ae25fc4ca4d2382e2179998134182/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:43.392670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e8da336a5f4a5c8d5d4ea5bfcfa639, user_id=076ae25fc4ca4d2382e2179998134182, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:43.398175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:43.564950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1201/2390] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:43 2022] PUT /identity/v3/projects/f8e8da336a5f4a5c8d5d4ea5bfcfa639/users/076ae25fc4ca4d2382e2179998134182/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:43.576285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f337d-a13d-492c-afc5-7ef550f2e955 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:43.576743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f337d-a13d-492c-afc5-7ef550f2e955 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:43.576845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f337d-a13d-492c-afc5-7ef550f2e955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:43.683641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0c3f337d-a13d-492c-afc5-7ef550f2e955 None None] MFA Rules not processed for user `076ae25fc4ca4d2382e2179998134182`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:43.688388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0c3f337d-a13d-492c-afc5-7ef550f2e955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:43.879368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:43.880793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1190/2391] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:43.882074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:44.010527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc02333-8a6f-4078-95b2-e63bb83028dd admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:44.015899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa059e-cb3b-4d75-b7eb-27b04cb5df6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:44.016592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa059e-cb3b-4d75-b7eb-27b04cb5df6c service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:44.016929 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa059e-cb3b-4d75-b7eb-27b04cb5df6c service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:44.017402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa059e-cb3b-4d75-b7eb-27b04cb5df6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:44.032616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa059e-cb3b-4d75-b7eb-27b04cb5df6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:44.034396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa059e-cb3b-4d75-b7eb-27b04cb5df6c service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:44.055815 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1202/2392] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:45.843521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:48.144593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:49.596764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f337d-a13d-492c-afc5-7ef550f2e955 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:49.745912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f337d-a13d-492c-afc5-7ef550f2e955 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:49.765714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:49.768300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:49.770919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:49.770919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] PATH_INFO: `/v3/users/076ae25fc4ca4d2382e2179998134182` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:49.773358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=076ae25fc4ca4d2382e2179998134182)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:49.778367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:50.064332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '076ae25fc4ca4d2382e2179998134182'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:50.157407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '076ae25fc4ca4d2382e2179998134182'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:50.159173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '076ae25fc4ca4d2382e2179998134182'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:50.159173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '076ae25fc4ca4d2382e2179998134182'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:50.175775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '076ae25fc4ca4d2382e2179998134182'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:50.213096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1191/2393] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:49 2022] DELETE /identity/v3/users/076ae25fc4ca4d2382e2179998134182 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:50.215134 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10832328-2ad9-47fb-bb70-0b30d2152e97 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:50.216018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10832328-2ad9-47fb-bb70-0b30d2152e97 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:50.216936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10832328-2ad9-47fb-bb70-0b30d2152e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:50.244598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-10832328-2ad9-47fb-bb70-0b30d2152e97 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:50.246352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-10832328-2ad9-47fb-bb70-0b30d2152e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:50.354025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1203/2394] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:50.364902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:50.366733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:50.433373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b739c-fca2-4bab-9b58-96697ef29c62 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:50.437067 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a84529-67ea-45af-8e79-968ba9512a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:50.437681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a84529-67ea-45af-8e79-968ba9512a6a service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:50.437958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a84529-67ea-45af-8e79-968ba9512a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:50.438273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a84529-67ea-45af-8e79-968ba9512a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:50.449993 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a84529-67ea-45af-8e79-968ba9512a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:50.451537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a84529-67ea-45af-8e79-968ba9512a6a service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:50.463237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1192/2395] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:50.806693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:51.105151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10832328-2ad9-47fb-bb70-0b30d2152e97 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:51.105963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-10832328-2ad9-47fb-bb70-0b30d2152e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:51.128956 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10832328-2ad9-47fb-bb70-0b30d2152e97 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:51.140592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:51.141438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:51.141832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:51.142182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] PATH_INFO: `/v3/projects/f8e8da336a5f4a5c8d5d4ea5bfcfa639` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:51.143814 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8e8da336a5f4a5c8d5d4ea5bfcfa639)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:51.146026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:51.292249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8e8da336a5f4a5c8d5d4ea5bfcfa639', 'request_id': 'req-ecbcfb66-f387-4257-9900-703b980a6f22'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:51.301243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8e8da336a5f4a5c8d5d4ea5bfcfa639', 'request_id': 'req-ecbcfb66-f387-4257-9900-703b980a6f22'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:51.408465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8e8da336a5f4a5c8d5d4ea5bfcfa639', 'request_id': 'req-ecbcfb66-f387-4257-9900-703b980a6f22'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:51.421698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] The token cache is being invalidate because project f8e8da336a5f4a5c8d5d4ea5bfcfa639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:51.423544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:51.423544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ecbcfb66-f387-4257-9900-703b980a6f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:51.428138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1204/2396] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:51 2022] DELETE /identity/v3/projects/f8e8da336a5f4a5c8d5d4ea5bfcfa639 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:51.434386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c719c8-cfb2-4ab9-9cec-dbc701cdf5f5 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:51.435308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c719c8-cfb2-4ab9-9cec-dbc701cdf5f5 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:51.435862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c719c8-cfb2-4ab9-9cec-dbc701cdf5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:51.458709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-92c719c8-cfb2-4ab9-9cec-dbc701cdf5f5 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:51.460793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-92c719c8-cfb2-4ab9-9cec-dbc701cdf5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:51.537039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1193/2397] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:51.546153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88c4f-4c1f-46ff-9452-207c7e0ee9b1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:51.546545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88c4f-4c1f-46ff-9452-207c7e0ee9b1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:51.546828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88c4f-4c1f-46ff-9452-207c7e0ee9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:51.569264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5ac88c4f-4c1f-46ff-9452-207c7e0ee9b1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:51.572634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5ac88c4f-4c1f-46ff-9452-207c7e0ee9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:51.578947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1205/2398] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:51.584120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c719c8-cfb2-4ab9-9cec-dbc701cdf5f5 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:51.595654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c719c8-cfb2-4ab9-9cec-dbc701cdf5f5 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:51.607230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:51.607230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:51.607230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:51.607230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:51.607936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:51.607936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:51.607936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:51.619297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1194/2399] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:56:51.624442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88c4f-4c1f-46ff-9452-207c7e0ee9b1 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:51.634608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88c4f-4c1f-46ff-9452-207c7e0ee9b1 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:51.639794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0705796-341e-4f4d-bc3e-842369537165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:51.641293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0705796-341e-4f4d-bc3e-842369537165 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:51.641293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0705796-341e-4f4d-bc3e-842369537165 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:51.641618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0705796-341e-4f4d-bc3e-842369537165 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:51.642389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0705796-341e-4f4d-bc3e-842369537165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:51.644420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0705796-341e-4f4d-bc3e-842369537165 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:51.813297 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1206/2400] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:51 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:51.817936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:51.828821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5903b-6da7-4401-bcac-55450df23226 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:51.888469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d776377-9e07-47d9-b582-0ce150c543b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:51.888907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d776377-9e07-47d9-b582-0ce150c543b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:51.889197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d776377-9e07-47d9-b582-0ce150c543b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:51.889473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d776377-9e07-47d9-b582-0ce150c543b4 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:51.890124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d776377-9e07-47d9-b582-0ce150c543b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:51.892110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d776377-9e07-47d9-b582-0ce150c543b4 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:52.029982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1195/2401] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:51 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:52.035197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0705796-341e-4f4d-bc3e-842369537165 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:52.046559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0705796-341e-4f4d-bc3e-842369537165 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:52.052852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9a94e-f4f7-4387-ade5-70a5451b8fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:52.054584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9a94e-f4f7-4387-ade5-70a5451b8fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:52.054877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9a94e-f4f7-4387-ade5-70a5451b8fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:52.055277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9a94e-f4f7-4387-ade5-70a5451b8fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:52.056031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9a94e-f4f7-4387-ade5-70a5451b8fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:52.057903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9a94e-f4f7-4387-ade5-70a5451b8fc5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:52.068862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1207/2402] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:52.073887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d776377-9e07-47d9-b582-0ce150c543b4 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:52.084039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d776377-9e07-47d9-b582-0ce150c543b4 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:52.088990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f937042-370e-48de-b201-43d3103056ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:52.089599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f937042-370e-48de-b201-43d3103056ff admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:52.089947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f937042-370e-48de-b201-43d3103056ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:52.090273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f937042-370e-48de-b201-43d3103056ff admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:52.090925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f937042-370e-48de-b201-43d3103056ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:52.092764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f937042-370e-48de-b201-43d3103056ff admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:52.104955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1196/2403] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:52.108959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9a94e-f4f7-4387-ade5-70a5451b8fc5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:52.119961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9a94e-f4f7-4387-ade5-70a5451b8fc5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:52.124821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:52.125594 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:52.125912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:52.126220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] PATH_INFO: `/v3/projects/179292c42f8e4041adc935c80dba0f70/users/1f10a95e49744a6dbb73e54e00d81add/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:52.141846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=179292c42f8e4041adc935c80dba0f70, user_id=1f10a95e49744a6dbb73e54e00d81add, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:52.144503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:52.249542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1208/2404] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:52 2022] PUT /identity/v3/projects/179292c42f8e4041adc935c80dba0f70/users/1f10a95e49744a6dbb73e54e00d81add/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:52.254862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f57a-fade-4858-a648-60735e303cee None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:52.255154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f57a-fade-4858-a648-60735e303cee None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:52.255429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f57a-fade-4858-a648-60735e303cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:52.276808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-80f2f57a-fade-4858-a648-60735e303cee None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:52.278054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-80f2f57a-fade-4858-a648-60735e303cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:52.357035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1197/2405] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:52.369331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:52.371318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:52.476093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2e13b-df56-48e4-ac48-a1e0972d0cec admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:52.483085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6c2a6-f299-492c-9b04-3b4dea97ea21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:52.484042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6c2a6-f299-492c-9b04-3b4dea97ea21 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:52.484328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6c2a6-f299-492c-9b04-3b4dea97ea21 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:52.484609 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6c2a6-f299-492c-9b04-3b4dea97ea21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:52.497896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6c2a6-f299-492c-9b04-3b4dea97ea21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:52.499630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6c2a6-f299-492c-9b04-3b4dea97ea21 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:52.510914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1209/2406] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:52.914755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:54.363159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:55.878036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:00.112188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca990-1764-462a-bd64-072e8c875a65 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:00.113228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca990-1764-462a-bd64-072e8c875a65 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:00.113663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca990-1764-462a-bd64-072e8c875a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:00.157318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2b0ca990-1764-462a-bd64-072e8c875a65 None None] MFA Rules not processed for user `1f10a95e49744a6dbb73e54e00d81add`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:00.159828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2b0ca990-1764-462a-bd64-072e8c875a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:00.260724 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1198/2407] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:57:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:57:00.265473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6c2a6-f299-492c-9b04-3b4dea97ea21 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:00.278563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6c2a6-f299-492c-9b04-3b4dea97ea21 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:00.285253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487083cc-02b5-46b6-8f55-488fee5fe04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:00.285981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487083cc-02b5-46b6-8f55-488fee5fe04b admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:00.286296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487083cc-02b5-46b6-8f55-488fee5fe04b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:00.286576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487083cc-02b5-46b6-8f55-488fee5fe04b admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:00.287240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487083cc-02b5-46b6-8f55-488fee5fe04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:00.290352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487083cc-02b5-46b6-8f55-488fee5fe04b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:00.408790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1210/2408] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:57:00 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:57:00.416928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca990-1764-462a-bd64-072e8c875a65 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:00.430864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca990-1764-462a-bd64-072e8c875a65 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:00.495919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7aa33b-b873-458b-afc1-c399dc9bcc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:00.496633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aa33b-b873-458b-afc1-c399dc9bcc1c admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:00.496982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aa33b-b873-458b-afc1-c399dc9bcc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:00.497431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aa33b-b873-458b-afc1-c399dc9bcc1c admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:00.497891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7aa33b-b873-458b-afc1-c399dc9bcc1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:00.499943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7aa33b-b873-458b-afc1-c399dc9bcc1c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:00.651191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1199/2409] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:57:00 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:57:00.655507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487083cc-02b5-46b6-8f55-488fee5fe04b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:00.669332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487083cc-02b5-46b6-8f55-488fee5fe04b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:00.675672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:00.676766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:00.677056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:00.677326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:00.678653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:00.680800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:00.702949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1211/2410] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:57:00.705361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7aa33b-b873-458b-afc1-c399dc9bcc1c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:00.717790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7aa33b-b873-458b-afc1-c399dc9bcc1c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:00.739018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a477a3-26fa-4dce-8593-26ee47333719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:00.739723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a477a3-26fa-4dce-8593-26ee47333719 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:00.740010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a477a3-26fa-4dce-8593-26ee47333719 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:00.740293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a477a3-26fa-4dce-8593-26ee47333719 admin admin] PATH_INFO: `/v3/projects/f8ded52ce16d4292983b3e8159d63c03/users/21e143acdc0d415b8575c22c7a320572/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:00.745673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:00.747820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:00.762916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631df8-73f1-47fd-abd8-f6aed3062def admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:00.783072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:00.783710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:00.784385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:00.784747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:00.785106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] PATH_INFO: `/v3/users/a7f325f254994077a62b744fb4262709` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:00.786553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f325f254994077a62b744fb4262709)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:00.789175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:00.802891 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a477a3-26fa-4dce-8593-26ee47333719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ded52ce16d4292983b3e8159d63c03, user_id=21e143acdc0d415b8575c22c7a320572, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:00.809225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a477a3-26fa-4dce-8593-26ee47333719 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:00.810625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:00.917493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1200/2411] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:57:00 2022] PUT /identity/v3/projects/f8ded52ce16d4292983b3e8159d63c03/users/21e143acdc0d415b8575c22c7a320572/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:57:00.920185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a477a3-26fa-4dce-8593-26ee47333719 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:00.987952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a477a3-26fa-4dce-8593-26ee47333719 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:00.991658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cd4e8-1485-4f31-9d1e-ae12225a5a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:00.992278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cd4e8-1485-4f31-9d1e-ae12225a5a52 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:00.992556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cd4e8-1485-4f31-9d1e-ae12225a5a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:00.992839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cd4e8-1485-4f31-9d1e-ae12225a5a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:00.993606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cd4e8-1485-4f31-9d1e-ae12225a5a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:00.995206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cd4e8-1485-4f31-9d1e-ae12225a5a52 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:01.006408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1201/2412] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:57:01.010709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cd4e8-1485-4f31-9d1e-ae12225a5a52 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:01.022129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cd4e8-1485-4f31-9d1e-ae12225a5a52 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:01.026709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f3836-8ee3-4f37-a9af-5b0d67ea56ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:01.027416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f3836-8ee3-4f37-a9af-5b0d67ea56ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:01.027693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f3836-8ee3-4f37-a9af-5b0d67ea56ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:01.027959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f3836-8ee3-4f37-a9af-5b0d67ea56ff admin admin] PATH_INFO: `/v3/projects/f8ded52ce16d4292983b3e8159d63c03/users/21e143acdc0d415b8575c22c7a320572/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:01.029894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f3836-8ee3-4f37-a9af-5b0d67ea56ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ded52ce16d4292983b3e8159d63c03, user_id=21e143acdc0d415b8575c22c7a320572, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:01.032883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f3836-8ee3-4f37-a9af-5b0d67ea56ff admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:01.044168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1202/2413] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:57:01 2022] PUT /identity/v3/projects/f8ded52ce16d4292983b3e8159d63c03/users/21e143acdc0d415b8575c22c7a320572/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:57:01.059087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a9f6-d857-44a5-81af-201b1e5701a4 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:01.059450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a9f6-d857-44a5-81af-201b1e5701a4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:01.059803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a9f6-d857-44a5-81af-201b1e5701a4 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:01.061751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1203/2414] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 8 13:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:57:01.077344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96a26c-9cb5-45ec-b862-d16d9ea2039e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:01.078802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96a26c-9cb5-45ec-b862-d16d9ea2039e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:01.079100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96a26c-9cb5-45ec-b862-d16d9ea2039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:01.095693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f325f254994077a62b744fb4262709'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:01.110415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9f96a26c-9cb5-45ec-b862-d16d9ea2039e None None] MFA Rules not processed for user `645282ae0ac7450dbd27d71c467b5216`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:01.111402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9f96a26c-9cb5-45ec-b862-d16d9ea2039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:01.179500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1204/2415] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 8 13:57:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:01.191433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96a26c-9cb5-45ec-b862-d16d9ea2039e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:01.202836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96a26c-9cb5-45ec-b862-d16d9ea2039e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:01.208617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:01.209426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:01.209708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:01.211505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:01.213612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:01.226227 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f325f254994077a62b744fb4262709'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:01.229645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f325f254994077a62b744fb4262709'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:01.238214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:01.242532 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:01.244958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f325f254994077a62b744fb4262709'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:01.256094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-749df78c-26d0-4009-b8a0-2622dbe49eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f325f254994077a62b744fb4262709'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:01.264689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1212/2417] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:57:00 2022] DELETE /identity/v3/users/a7f325f254994077a62b744fb4262709 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:57:01.266245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1205/2416] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:57:01.270386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f428e-f801-48fb-963a-a2593a9da148 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:01.270860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f428e-f801-48fb-963a-a2593a9da148 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:01.271188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f428e-f801-48fb-963a-a2593a9da148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:01.304212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-8c1f428e-f801-48fb-963a-a2593a9da148 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:01.305972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8c1f428e-f801-48fb-963a-a2593a9da148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:01.380983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1213/2418] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:57:01.392514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:01.466060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb40ce9-af0d-4dcf-9716-94649f7ff5ef service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:01.557810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286e561-eb93-4d2c-b3f3-8e4141a2d0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:01.558495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e561-eb93-4d2c-b3f3-8e4141a2d0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:01.558783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e561-eb93-4d2c-b3f3-8e4141a2d0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:01.559069 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e561-eb93-4d2c-b3f3-8e4141a2d0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:01.570219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286e561-eb93-4d2c-b3f3-8e4141a2d0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:01.571747 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286e561-eb93-4d2c-b3f3-8e4141a2d0e7 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:01.584035 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1206/2419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:02.441275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:02.563786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f428e-f801-48fb-963a-a2593a9da148 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:02.568654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8c1f428e-f801-48fb-963a-a2593a9da148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:02.610788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f428e-f801-48fb-963a-a2593a9da148 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:02.630428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:02.631503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:02.632526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:02.633104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] PATH_INFO: `/v3/projects/64b7f483a40b4510b046abbe5f5f06a7` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:02.635341 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b7f483a40b4510b046abbe5f5f06a7)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:02.641719 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:02.939685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b7f483a40b4510b046abbe5f5f06a7', 'request_id': 'req-9f9ef9ae-81ce-4bec-9777-533e53f9be43'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:02.951275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b7f483a40b4510b046abbe5f5f06a7', 'request_id': 'req-9f9ef9ae-81ce-4bec-9777-533e53f9be43'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:03.101212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b7f483a40b4510b046abbe5f5f06a7', 'request_id': 'req-9f9ef9ae-81ce-4bec-9777-533e53f9be43'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:03.112771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] The token cache is being invalidate because project 64b7f483a40b4510b046abbe5f5f06a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:57:03.113066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:03.113727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-9f9ef9ae-81ce-4bec-9777-533e53f9be43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:03.117844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1214/2420] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:57:02 2022] DELETE /identity/v3/projects/64b7f483a40b4510b046abbe5f5f06a7 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:57:03.120858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95043b05-7e00-49b9-9287-be53dc3a87ef None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:03.121145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95043b05-7e00-49b9-9287-be53dc3a87ef None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:03.121624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95043b05-7e00-49b9-9287-be53dc3a87ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:03.143843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-95043b05-7e00-49b9-9287-be53dc3a87ef None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:03.145181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-95043b05-7e00-49b9-9287-be53dc3a87ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:03.235845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1207/2421] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:03.244210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56286d3e-be3a-4519-b796-c91c9713823e None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:03.244788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56286d3e-be3a-4519-b796-c91c9713823e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:03.245219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56286d3e-be3a-4519-b796-c91c9713823e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:03.269168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-56286d3e-be3a-4519-b796-c91c9713823e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:03.270759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-56286d3e-be3a-4519-b796-c91c9713823e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:03.282437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1215/2422] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:03.286751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95043b05-7e00-49b9-9287-be53dc3a87ef None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:03.298921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95043b05-7e00-49b9-9287-be53dc3a87ef None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:03.305815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:03.306739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:03.307266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:03.307934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:03.308888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:57:03.309397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:03.311739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:03.330866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1208/2423] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:57:03.335640 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56286d3e-be3a-4519-b796-c91c9713823e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:03.350460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56286d3e-be3a-4519-b796-c91c9713823e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:03.357940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d137389-aa53-4b68-8689-2b0f3ac9ed17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:03.358837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d137389-aa53-4b68-8689-2b0f3ac9ed17 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:03.359280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d137389-aa53-4b68-8689-2b0f3ac9ed17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:03.359743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d137389-aa53-4b68-8689-2b0f3ac9ed17 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:03.360563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d137389-aa53-4b68-8689-2b0f3ac9ed17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:03.363313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d137389-aa53-4b68-8689-2b0f3ac9ed17 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:03.520425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1216/2424] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:57:03 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:57:03.525542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:03.543641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f637ece-7f76-442b-b520-55033c64a20f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:03.630183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:03.757227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0f123-74f3-4427-a238-93fa4197b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:03.757227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0f123-74f3-4427-a238-93fa4197b655 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:03.757227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0f123-74f3-4427-a238-93fa4197b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:03.757227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0f123-74f3-4427-a238-93fa4197b655 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:03.757812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0f123-74f3-4427-a238-93fa4197b655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:03.760616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0f123-74f3-4427-a238-93fa4197b655 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:03.914631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1209/2425] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:57:03 2022] POST /identity/v3/users => generated 285 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:57:03.918674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d137389-aa53-4b68-8689-2b0f3ac9ed17 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:03.935991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d137389-aa53-4b68-8689-2b0f3ac9ed17 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:03.954308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1284e77-f061-4e14-92dc-fdd0d8346bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:03.955178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1284e77-f061-4e14-92dc-fdd0d8346bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:03.955603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1284e77-f061-4e14-92dc-fdd0d8346bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:03.956049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1284e77-f061-4e14-92dc-fdd0d8346bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:03.956852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1284e77-f061-4e14-92dc-fdd0d8346bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:03.962059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1284e77-f061-4e14-92dc-fdd0d8346bc3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:03.993482 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1217/2426] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:57:03.995156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0f123-74f3-4427-a238-93fa4197b655 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:04.033213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0f123-74f3-4427-a238-93fa4197b655 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:04.052702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8091813-5b35-4e74-a116-f418f774e196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:04.054506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8091813-5b35-4e74-a116-f418f774e196 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:04.054975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8091813-5b35-4e74-a116-f418f774e196 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:04.061455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8091813-5b35-4e74-a116-f418f774e196 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:04.061455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8091813-5b35-4e74-a116-f418f774e196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:04.062528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8091813-5b35-4e74-a116-f418f774e196 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:04.091446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1210/2427] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:57:04.098759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1284e77-f061-4e14-92dc-fdd0d8346bc3 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:04.112452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1284e77-f061-4e14-92dc-fdd0d8346bc3 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:04.128492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:04.129179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:04.129460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:04.129735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] PATH_INFO: `/v3/projects/ffca266b071041528727f1abf04b27bf/users/b4c71dd9ab3b47d4b1f6c100792be0fe/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:04.170197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffca266b071041528727f1abf04b27bf, user_id=b4c71dd9ab3b47d4b1f6c100792be0fe, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:04.174062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:04.356936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1218/2428] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:57:04 2022] PUT /identity/v3/projects/ffca266b071041528727f1abf04b27bf/users/b4c71dd9ab3b47d4b1f6c100792be0fe/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:57:04.367244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc047ab-5691-47e3-90fb-2a5d4e6ed26c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:04.367536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc047ab-5691-47e3-90fb-2a5d4e6ed26c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:04.368064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc047ab-5691-47e3-90fb-2a5d4e6ed26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:04.438889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ecc047ab-5691-47e3-90fb-2a5d4e6ed26c None None] MFA Rules not processed for user `b4c71dd9ab3b47d4b1f6c100792be0fe`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:04.440188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ecc047ab-5691-47e3-90fb-2a5d4e6ed26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:04.450481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:04.557063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1211/2429] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:57:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:04.570966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:04.573100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:04.678164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113460ee-964d-4037-a786-c2877b46ca6a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:04.685367 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a751fdb-447a-433e-9622-345e9a9b3ae4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c07c85b63514d10bb17b836c60f2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:04.686271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751fdb-447a-433e-9622-345e9a9b3ae4 service glance] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:04.686686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751fdb-447a-433e-9622-345e9a9b3ae4 service glance] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:04.687004 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751fdb-447a-433e-9622-345e9a9b3ae4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:04.699728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a751fdb-447a-433e-9622-345e9a9b3ae4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:04.701268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a751fdb-447a-433e-9622-345e9a9b3ae4 service glance] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:04.720259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1219/2430] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:57:07.021166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b2d54-11a7-4573-9a46-c629123d9b95 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:07.025705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b2d54-11a7-4573-9a46-c629123d9b95 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:07.025705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b2d54-11a7-4573-9a46-c629123d9b95 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:07.025705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1212/2431] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:57:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:07.036727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ff8be-87e8-44f3-a625-86e7406c0424 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:07.037254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ff8be-87e8-44f3-a625-86e7406c0424 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:07.037254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ff8be-87e8-44f3-a625-86e7406c0424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:07.102684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a72ff8be-87e8-44f3-a625-86e7406c0424 None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:07.107251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a72ff8be-87e8-44f3-a625-86e7406c0424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:07.208115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1220/2432] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:57:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:57:09.956124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062589e8-2174-4c0c-9522-e5da5a00bd22 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:09.958681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062589e8-2174-4c0c-9522-e5da5a00bd22 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:09.958681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062589e8-2174-4c0c-9522-e5da5a00bd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:09.993520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-062589e8-2174-4c0c-9522-e5da5a00bd22 None None] MFA Rules not processed for user `21e143acdc0d415b8575c22c7a320572`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:09.998987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-062589e8-2174-4c0c-9522-e5da5a00bd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:10.083179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1213/2433] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:10.692326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8d9a-96c4-431d-b439-e0727b47e87f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:10.693300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8d9a-96c4-431d-b439-e0727b47e87f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:10.693300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8d9a-96c4-431d-b439-e0727b47e87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:10.713651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0a4a8d9a-96c4-431d-b439-e0727b47e87f None None] MFA Rules not processed for user `772205d93ce047e9ad4ddefadec300f4`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:10.715197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0a4a8d9a-96c4-431d-b439-e0727b47e87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:10.760181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:10.778451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1221/2434] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:57:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:17.823283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062589e8-2174-4c0c-9522-e5da5a00bd22 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:17.824705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-062589e8-2174-4c0c-9522-e5da5a00bd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:17.838610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062589e8-2174-4c0c-9522-e5da5a00bd22 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:17.842913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7cc89-0eae-47dc-bc01-fccb1f0d1663 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c07c85b63514d10bb17b836c60f2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:17.843704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7cc89-0eae-47dc-bc01-fccb1f0d1663 service glance] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:17.844049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7cc89-0eae-47dc-bc01-fccb1f0d1663 service glance] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:17.849026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7cc89-0eae-47dc-bc01-fccb1f0d1663 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:17.856595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7cc89-0eae-47dc-bc01-fccb1f0d1663 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:17.858111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7cc89-0eae-47dc-bc01-fccb1f0d1663 service glance] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:17.871101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1214/2435] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:17.927550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8d9a-96c4-431d-b439-e0727b47e87f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:17.939433 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8d9a-96c4-431d-b439-e0727b47e87f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:17.947296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4dfc3-ec03-425f-b574-466c874c47ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c07c85b63514d10bb17b836c60f2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:17.948615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4dfc3-ec03-425f-b574-466c874c47ff service glance] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:17.949277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4dfc3-ec03-425f-b574-466c874c47ff service glance] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:17.949827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4dfc3-ec03-425f-b574-466c874c47ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:17.952250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-06e4dfc3-ec03-425f-b574-466c874c47ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:18.024614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4dfc3-ec03-425f-b574-466c874c47ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:18.026575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4dfc3-ec03-425f-b574-466c874c47ff service glance] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:18.037215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1222/2436] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:57:18.374001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18219-825e-4461-a48d-8db48eaa4134 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:18.374503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18219-825e-4461-a48d-8db48eaa4134 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:18.375022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18219-825e-4461-a48d-8db48eaa4134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:18.399653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-bbf18219-825e-4461-a48d-8db48eaa4134 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:18.401888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bbf18219-825e-4461-a48d-8db48eaa4134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:18.463588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad043-21ed-40d3-8fa2-09d6ddbb3aa3 None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:18.463988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad043-21ed-40d3-8fa2-09d6ddbb3aa3 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:18.465870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad043-21ed-40d3-8fa2-09d6ddbb3aa3 None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:18.467880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1223/2437] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:57:18 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:18.473365 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a139-7535-4596-b5eb-a8d8d14471ea None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:18.473667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a139-7535-4596-b5eb-a8d8d14471ea None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:18.473903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a139-7535-4596-b5eb-a8d8d14471ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:18.486171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1215/2438] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:18.500975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18219-825e-4461-a48d-8db48eaa4134 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:18.502748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a367a139-7535-4596-b5eb-a8d8d14471ea None None] MFA Rules not processed for user `a71d353c1bd849028930261e9306fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:18.502901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bbf18219-825e-4461-a48d-8db48eaa4134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:18.504013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a367a139-7535-4596-b5eb-a8d8d14471ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:18.548766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:18.574559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1224/2439] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:18.589753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367a139-7535-4596-b5eb-a8d8d14471ea None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:18.591239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a367a139-7535-4596-b5eb-a8d8d14471ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:18.603195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367a139-7535-4596-b5eb-a8d8d14471ea None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:18.614694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a39f74-451f-40b4-9fbc-9a4039cd4a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:18.617049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a39f74-451f-40b4-9fbc-9a4039cd4a00 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:18.617705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a39f74-451f-40b4-9fbc-9a4039cd4a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:18.618405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a39f74-451f-40b4-9fbc-9a4039cd4a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:18.619667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18219-825e-4461-a48d-8db48eaa4134 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:18.634603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9038f9d-0097-4d7d-9f61-44a49281e128 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:18.635495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9038f9d-0097-4d7d-9f61-44a49281e128 service swift] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:18.635896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9038f9d-0097-4d7d-9f61-44a49281e128 service swift] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:18.636129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a39f74-451f-40b4-9fbc-9a4039cd4a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:18.636538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9038f9d-0097-4d7d-9f61-44a49281e128 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:18.637764 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a39f74-451f-40b4-9fbc-9a4039cd4a00 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:18.648855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1225/2440] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:57:18.651789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9038f9d-0097-4d7d-9f61-44a49281e128 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:18.656111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9038f9d-0097-4d7d-9f61-44a49281e128 service swift] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:18.664308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1216/2441] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 13:57:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 13:57:18.780144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908a43-7982-4adc-989a-b3718ca8f08c None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:18.780701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908a43-7982-4adc-989a-b3718ca8f08c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:18.780701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908a43-7982-4adc-989a-b3718ca8f08c None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:18.782162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1226/2442] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 13:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:18.788055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f539c96-ebfb-4bc2-9b6b-9e86226c38ca None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:18.788345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f539c96-ebfb-4bc2-9b6b-9e86226c38ca None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:18.788625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f539c96-ebfb-4bc2-9b6b-9e86226c38ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:18.823569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9f539c96-ebfb-4bc2-9b6b-9e86226c38ca None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:18.824750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9f539c96-ebfb-4bc2-9b6b-9e86226c38ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:18.835167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1217/2443] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:18.850477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf908a43-7982-4adc-989a-b3718ca8f08c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:18.864656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf908a43-7982-4adc-989a-b3718ca8f08c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:18.869228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7b613-666f-4581-bae8-bb20b37cb84f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:18.872262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7b613-666f-4581-bae8-bb20b37cb84f service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:18.872555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7b613-666f-4581-bae8-bb20b37cb84f service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:18.872834 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7b613-666f-4581-bae8-bb20b37cb84f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:18.891736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7b613-666f-4581-bae8-bb20b37cb84f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:18.893796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7b613-666f-4581-bae8-bb20b37cb84f service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:18.908474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1227/2444] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:28.803886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=23 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:28.903617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=69 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:30.231917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3aca-345a-4194-8f94-6d8fbf53d144 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:30.232481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3aca-345a-4194-8f94-6d8fbf53d144 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:30.232481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3aca-345a-4194-8f94-6d8fbf53d144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:30.256259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-be3f3aca-345a-4194-8f94-6d8fbf53d144 None None] MFA Rules not processed for user `1f10a95e49744a6dbb73e54e00d81add`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:30.260719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-be3f3aca-345a-4194-8f94-6d8fbf53d144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:30.272146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1218/2445] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:57:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:30.392938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2ea6-6964-4645-ab99-ba8e9dbf2d1f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:30.393519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2ea6-6964-4645-ab99-ba8e9dbf2d1f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:30.393519 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2ea6-6964-4645-ab99-ba8e9dbf2d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:30.417580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f57b2ea6-6964-4645-ab99-ba8e9dbf2d1f None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:30.418626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f57b2ea6-6964-4645-ab99-ba8e9dbf2d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:30.432778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1228/2446] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:57:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:30.441673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3aca-345a-4194-8f94-6d8fbf53d144 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:30.458579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3aca-345a-4194-8f94-6d8fbf53d144 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:30.467214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2b7c9-6dcd-43a8-9aea-dc6ca75c2c3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:30.467917 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2b7c9-6dcd-43a8-9aea-dc6ca75c2c3f service swift] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:30.468154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2b7c9-6dcd-43a8-9aea-dc6ca75c2c3f service swift] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:30.468392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2b7c9-6dcd-43a8-9aea-dc6ca75c2c3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:30.484071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2b7c9-6dcd-43a8-9aea-dc6ca75c2c3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:30.486344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2b7c9-6dcd-43a8-9aea-dc6ca75c2c3f service swift] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:30.499840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1219/2447] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 13:57:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:57:40.410981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:40.494037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=9 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:48.767510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b2ea6-6964-4645-ab99-ba8e9dbf2d1f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:48.778598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b2ea6-6964-4645-ab99-ba8e9dbf2d1f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:48.783038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:48.783183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:48.783475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:48.783806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:48.785793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:48.869397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:48.870971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:48.881153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1229/2448] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:50.671150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7179-6cd2-41fc-83d1-e3ee67952c95 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:50.672197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7179-6cd2-41fc-83d1-e3ee67952c95 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:50.672825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7179-6cd2-41fc-83d1-e3ee67952c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:50.715514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-967a7179-6cd2-41fc-83d1-e3ee67952c95 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:50.716772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-967a7179-6cd2-41fc-83d1-e3ee67952c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:50.732183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1220/2449] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 13:57:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:50.758619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:50.771057 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5630f-1970-40ee-83dd-bf6981f160a3 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:50.777448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec100d-cb3e-497b-a4f5-a6dbf09c8545 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:50.780066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec100d-cb3e-497b-a4f5-a6dbf09c8545 service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:50.780066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec100d-cb3e-497b-a4f5-a6dbf09c8545 service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:50.780066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec100d-cb3e-497b-a4f5-a6dbf09c8545 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:50.792059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec100d-cb3e-497b-a4f5-a6dbf09c8545 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:50.793672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec100d-cb3e-497b-a4f5-a6dbf09c8545 service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:50.803679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1230/2450] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 13:57:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:58:00.698120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:00.799653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=36 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:04.823967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a7179-6cd2-41fc-83d1-e3ee67952c95 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:04.890583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a7179-6cd2-41fc-83d1-e3ee67952c95 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:04.895665 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:04.896305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:04.896585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:04.896860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] PATH_INFO: `/v3/users/b4c71dd9ab3b47d4b1f6c100792be0fe` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:04.898000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4c71dd9ab3b47d4b1f6c100792be0fe)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:04.900200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:05.098935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4c71dd9ab3b47d4b1f6c100792be0fe'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.157997 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4c71dd9ab3b47d4b1f6c100792be0fe'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.158655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4c71dd9ab3b47d4b1f6c100792be0fe'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.159260 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4c71dd9ab3b47d4b1f6c100792be0fe'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.167574 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4c71dd9ab3b47d4b1f6c100792be0fe'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.188992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1221/2451] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:58:04 2022] DELETE /identity/v3/users/b4c71dd9ab3b47d4b1f6c100792be0fe => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:05.190725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea2aa8-bafa-4dc5-a4ce-cbe4568d7f1d None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:05.191232 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea2aa8-bafa-4dc5-a4ce-cbe4568d7f1d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:05.191349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea2aa8-bafa-4dc5-a4ce-cbe4568d7f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:05.211043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f2ea2aa8-bafa-4dc5-a4ce-cbe4568d7f1d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:05.212296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f2ea2aa8-bafa-4dc5-a4ce-cbe4568d7f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:05.273442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1231/2452] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:05.283516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:05.287064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:05.345491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaf9a6-5e29-4d1d-834a-503085db20d7 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:05.349782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7b642-1eac-4aeb-983b-172f9d0c447c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:05.350421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7b642-1eac-4aeb-983b-172f9d0c447c service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:05.350702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7b642-1eac-4aeb-983b-172f9d0c447c service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:05.350972 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7b642-1eac-4aeb-983b-172f9d0c447c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:05.360815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7b642-1eac-4aeb-983b-172f9d0c447c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:05.362298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7b642-1eac-4aeb-983b-172f9d0c447c service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:05.372294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1222/2453] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:58:05.796390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea2aa8-bafa-4dc5-a4ce-cbe4568d7f1d None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:05.797838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f2ea2aa8-bafa-4dc5-a4ce-cbe4568d7f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:05.809954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea2aa8-bafa-4dc5-a4ce-cbe4568d7f1d None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:05.814998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:05.815637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:05.815945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:05.816195 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] PATH_INFO: `/v3/projects/ffca266b071041528727f1abf04b27bf` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:05.817326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffca266b071041528727f1abf04b27bf)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:05.819395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:05.939162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffca266b071041528727f1abf04b27bf', 'request_id': 'req-390c30b8-657b-400e-93ad-902b78c9f1db'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.945730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffca266b071041528727f1abf04b27bf', 'request_id': 'req-390c30b8-657b-400e-93ad-902b78c9f1db'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.015045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffca266b071041528727f1abf04b27bf', 'request_id': 'req-390c30b8-657b-400e-93ad-902b78c9f1db'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.028112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] The token cache is being invalidate because project ffca266b071041528727f1abf04b27bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:58:06.028976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.030196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-390c30b8-657b-400e-93ad-902b78c9f1db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.031819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1232/2454] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:58:05 2022] DELETE /identity/v3/projects/ffca266b071041528727f1abf04b27bf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:06.037347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76106558-4d44-4ba1-b373-8927edf828e3 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.037840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76106558-4d44-4ba1-b373-8927edf828e3 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.037947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76106558-4d44-4ba1-b373-8927edf828e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.059331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-76106558-4d44-4ba1-b373-8927edf828e3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:06.060550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-76106558-4d44-4ba1-b373-8927edf828e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:06.078950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:06.132040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1223/2455] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:06.140098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887f6d-4816-48cb-b63f-8cf7e14c8a15 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.140383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887f6d-4816-48cb-b63f-8cf7e14c8a15 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.140683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887f6d-4816-48cb-b63f-8cf7e14c8a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.162954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a9887f6d-4816-48cb-b63f-8cf7e14c8a15 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:06.164180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a9887f6d-4816-48cb-b63f-8cf7e14c8a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:06.172281 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1233/2456] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:06.176812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76106558-4d44-4ba1-b373-8927edf828e3 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.187652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76106558-4d44-4ba1-b373-8927edf828e3 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.192769 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.193527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.194117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.194600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.195332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:58:06.195707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.197921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.210717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1224/2457] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 13:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:58:06.215864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887f6d-4816-48cb-b63f-8cf7e14c8a15 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.225464 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887f6d-4816-48cb-b63f-8cf7e14c8a15 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.230308 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2c92d-b611-4ada-93b9-20cc2613ccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.230890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2c92d-b611-4ada-93b9-20cc2613ccf2 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.231169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2c92d-b611-4ada-93b9-20cc2613ccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.231443 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2c92d-b611-4ada-93b9-20cc2613ccf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.232084 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2c92d-b611-4ada-93b9-20cc2613ccf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.234048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2c92d-b611-4ada-93b9-20cc2613ccf2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.353734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1234/2458] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 13:58:06 2022] POST /identity/v3/projects => generated 397 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:58:06.355198 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.367056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d466f-d30e-4a1c-b7ee-3cd71e72bb48 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.438130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7bc3c-c000-4a7b-a3f6-610589b31e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.438790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7bc3c-c000-4a7b-a3f6-610589b31e49 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.439071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7bc3c-c000-4a7b-a3f6-610589b31e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.439346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7bc3c-c000-4a7b-a3f6-610589b31e49 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.439981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7bc3c-c000-4a7b-a3f6-610589b31e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.441901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7bc3c-c000-4a7b-a3f6-610589b31e49 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.533507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1225/2459] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 13:58:06 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:58:06.534839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2c92d-b611-4ada-93b9-20cc2613ccf2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.545282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2c92d-b611-4ada-93b9-20cc2613ccf2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.554787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae59f73-8f77-4d56-a200-f20aa243e58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.554787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae59f73-8f77-4d56-a200-f20aa243e58f admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.554787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae59f73-8f77-4d56-a200-f20aa243e58f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.554787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae59f73-8f77-4d56-a200-f20aa243e58f admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.555215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae59f73-8f77-4d56-a200-f20aa243e58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.555215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae59f73-8f77-4d56-a200-f20aa243e58f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.563943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1235/2460] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:58:06.568082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7bc3c-c000-4a7b-a3f6-610589b31e49 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.578815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7bc3c-c000-4a7b-a3f6-610589b31e49 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.583440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5319b3e-307a-4f14-b097-5e6d756ccbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.584453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5319b3e-307a-4f14-b097-5e6d756ccbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.584743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5319b3e-307a-4f14-b097-5e6d756ccbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.585021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5319b3e-307a-4f14-b097-5e6d756ccbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.585661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5319b3e-307a-4f14-b097-5e6d756ccbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.587290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5319b3e-307a-4f14-b097-5e6d756ccbc3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.598071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1226/2461] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 13:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:58:06.602474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae59f73-8f77-4d56-a200-f20aa243e58f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.612907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae59f73-8f77-4d56-a200-f20aa243e58f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.617491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.618149 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.618575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.618856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] PATH_INFO: `/v3/projects/c502aaeedf2544358ed487cbd3b46df8/users/b4f786c2f4ac4654b6a83976e5abc8e1/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.634040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c502aaeedf2544358ed487cbd3b46df8, user_id=b4f786c2f4ac4654b6a83976e5abc8e1, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.637368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.732382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1236/2462] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 13:58:06 2022] PUT /identity/v3/projects/c502aaeedf2544358ed487cbd3b46df8/users/b4f786c2f4ac4654b6a83976e5abc8e1/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:06.743923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7d83-eeb5-4f49-889c-815a7e2f4c5c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.744737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7d83-eeb5-4f49-889c-815a7e2f4c5c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.745041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7d83-eeb5-4f49-889c-815a7e2f4c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.773737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6aee7d83-eeb5-4f49-889c-815a7e2f4c5c None None] MFA Rules not processed for user `b4f786c2f4ac4654b6a83976e5abc8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:06.775000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6aee7d83-eeb5-4f49-889c-815a7e2f4c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:06.839365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1227/2463] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:58:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:06.851029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.852726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:06.875637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:06.910557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0235-0eaf-4f11-83c5-ace6ff819a55 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.914168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e2b6f-f813-4d5b-8fb9-2e20b624644e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.914779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e2b6f-f813-4d5b-8fb9-2e20b624644e service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.915058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e2b6f-f813-4d5b-8fb9-2e20b624644e service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.915334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e2b6f-f813-4d5b-8fb9-2e20b624644e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.923969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e2b6f-f813-4d5b-8fb9-2e20b624644e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.925393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e2b6f-f813-4d5b-8fb9-2e20b624644e service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.935115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1237/2464] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:16.836462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:16.960748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=17 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:24.219258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7d83-eeb5-4f49-889c-815a7e2f4c5c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:24.286486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7d83-eeb5-4f49-889c-815a7e2f4c5c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:24.291688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:24.292405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:24.292686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:24.293503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] PATH_INFO: `/v3/users/b4f786c2f4ac4654b6a83976e5abc8e1` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:24.294720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f786c2f4ac4654b6a83976e5abc8e1)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:24.296799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:24.489757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f786c2f4ac4654b6a83976e5abc8e1'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:24.555888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f786c2f4ac4654b6a83976e5abc8e1'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:24.556462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f786c2f4ac4654b6a83976e5abc8e1'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:24.557079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f786c2f4ac4654b6a83976e5abc8e1'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:24.566194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f786c2f4ac4654b6a83976e5abc8e1'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:24.584411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1228/2465] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 13:58:24 2022] DELETE /identity/v3/users/b4f786c2f4ac4654b6a83976e5abc8e1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:24.589430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46345eb-817e-41b9-934e-ebbb0ce65f3f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:24.589921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46345eb-817e-41b9-934e-ebbb0ce65f3f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:24.590029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46345eb-817e-41b9-934e-ebbb0ce65f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:24.609707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c46345eb-817e-41b9-934e-ebbb0ce65f3f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:24.611019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c46345eb-817e-41b9-934e-ebbb0ce65f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:24.680005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1238/2466] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:24.686752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:24.688171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:24.761430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752f41d-e8b0-4ac1-836e-1cf53d0d064b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:24.765310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fa002-a04c-4cac-84da-0881c3ac85b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:24.765926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fa002-a04c-4cac-84da-0881c3ac85b7 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:24.766242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fa002-a04c-4cac-84da-0881c3ac85b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:24.766522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fa002-a04c-4cac-84da-0881c3ac85b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:24.777556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fa002-a04c-4cac-84da-0881c3ac85b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:24.779033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fa002-a04c-4cac-84da-0881c3ac85b7 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:24.793264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1229/2467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:25.330634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46345eb-817e-41b9-934e-ebbb0ce65f3f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:25.332261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c46345eb-817e-41b9-934e-ebbb0ce65f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:25.346808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46345eb-817e-41b9-934e-ebbb0ce65f3f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:25.352517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:25.353358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:25.353711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:25.354266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] PATH_INFO: `/v3/projects/c502aaeedf2544358ed487cbd3b46df8` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:25.355320 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c502aaeedf2544358ed487cbd3b46df8)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:25.357337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:25.517541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c502aaeedf2544358ed487cbd3b46df8', 'request_id': 'req-636dd1e4-e008-492c-b848-98cbab67c388'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:25.524644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c502aaeedf2544358ed487cbd3b46df8', 'request_id': 'req-636dd1e4-e008-492c-b848-98cbab67c388'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:25.615620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c502aaeedf2544358ed487cbd3b46df8', 'request_id': 'req-636dd1e4-e008-492c-b848-98cbab67c388'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:25.630356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] The token cache is being invalidate because project c502aaeedf2544358ed487cbd3b46df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:58:25.630644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:25.631278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:25.632773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1239/2468] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 13:58:25 2022] DELETE /identity/v3/projects/c502aaeedf2544358ed487cbd3b46df8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:26.900196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d36574-09de-46cf-b3dd-5d7e2315d4c6 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:26.900732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d36574-09de-46cf-b3dd-5d7e2315d4c6 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:26.901202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d36574-09de-46cf-b3dd-5d7e2315d4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:26.932145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-75d36574-09de-46cf-b3dd-5d7e2315d4c6 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:26.933317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-75d36574-09de-46cf-b3dd-5d7e2315d4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:26.962297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:27.035113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1230/2469] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 13:58:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:27.052717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:27.055106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:27.167159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dd1e4-e008-492c-b848-98cbab67c388 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:27.175136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:27.175136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:27.175136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:27.175136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:27.189647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:27.191959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:27.204755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1240/2470] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 13:58:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:58:29.657422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d36574-09de-46cf-b3dd-5d7e2315d4c6 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:29.660611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-75d36574-09de-46cf-b3dd-5d7e2315d4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:29.737702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d36574-09de-46cf-b3dd-5d7e2315d4c6 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:29.743921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:29.744778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:29.746001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:29.746496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:29.750882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:29.996113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:29.997507 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:30.010753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1231/2471] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:31.212922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:31.216557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:31.246111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f77c96-a787-457a-8c49-b58acefe63d1 service swift] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:31.253840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:31.254669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:31.255017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:31.255349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:31.257901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:31.282495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:31.285314 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:31.294761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:31.303733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1241/2472] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:40.004534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=69 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:44.491868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:44.493438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:44.507670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a5204-43e7-4960-a035-8b20da30ec14 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:44.513271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:44.513882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:44.518645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:44.518645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:44.518645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:44.576988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:44.578617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:44.587783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1232/2473] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:47.615986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:47.627395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f65aab-083e-4611-abcd-0cf9f9868a51 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:47.632798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:47.638015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:47.638015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:47.638015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:47.638015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:47.695170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:47.696542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:47.706017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1242/2474] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:54.583438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=28 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:57.701423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=28 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:12.015414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:12.026556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11373e-9381-4a77-8317-b6eab599d738 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:12.032713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:12.033450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:12.033704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:12.033919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:12.035954 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:12.095788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:12.097364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:12.108368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1233/2475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:22.102253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=28 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:39.828394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.830755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:39.849836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaee65b-73f2-44e5-9012-a3133aaa89a1 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.856396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.857124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.858225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.858573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.861981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:39.877026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.878817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.891817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1243/2476] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:41.167916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:41.179076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212a7c5-1956-4b0c-874b-e758fa026f83 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:41.183404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:41.189174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.189174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.189174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.189174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:41.200498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:41.204055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:41.215290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1234/2477] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:42.196404 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:42.207365 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442b330-94a0-4b3b-9621-006699dad435 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:42.211226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:42.212163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:42.212589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:42.213012 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:42.215092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:42.306206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:42.307319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:42.318637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1244/2478] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:42.542946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:42.558084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705665f-b6dd-41f7-a79b-a7adeaf1a133 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:42.567782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928419c-7c50-4847-8895-1657d6ac3acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:42.567782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928419c-7c50-4847-8895-1657d6ac3acd service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:42.567782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928419c-7c50-4847-8895-1657d6ac3acd service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:42.567782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928419c-7c50-4847-8895-1657d6ac3acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:42.570519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5928419c-7c50-4847-8895-1657d6ac3acd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:42.585006 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928419c-7c50-4847-8895-1657d6ac3acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:42.586847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928419c-7c50-4847-8895-1657d6ac3acd service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:42.600489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1235/2479] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:44.835114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:44.849554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843995e5-c1c1-4440-89ec-8daaf01cbad5 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:44.858619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:44.859420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:44.859773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:44.860136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:44.862052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:44.876439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:44.878745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:44.891561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1245/2480] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:44.955276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891c2be-bdb5-411e-83b5-8ff15d6a2867 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:44.955833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891c2be-bdb5-411e-83b5-8ff15d6a2867 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:44.955833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891c2be-bdb5-411e-83b5-8ff15d6a2867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:44.976840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c891c2be-bdb5-411e-83b5-8ff15d6a2867 None None] MFA Rules not processed for user `792808dcbf894aa98c671cf43ac21b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:44.978113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c891c2be-bdb5-411e-83b5-8ff15d6a2867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:44.991674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1236/2481] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:59:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:46.780671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:46.791622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f428e0-25a4-4d5f-be17-8c48b3dcdaf8 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.801129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.801129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.801129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.801129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.801697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:46.888991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:46.890592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:46.900737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1246/2482] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:52.522711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d6d5-73af-41ac-b78d-a6c5da737cf1 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.523362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d6d5-73af-41ac-b78d-a6c5da737cf1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.523362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d6d5-73af-41ac-b78d-a6c5da737cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.544001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-aa83d6d5-73af-41ac-b78d-a6c5da737cf1 None None] MFA Rules not processed for user `eca89f435299415c8448931e73f5b0fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:52.545583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-aa83d6d5-73af-41ac-b78d-a6c5da737cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.552674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1237/2483] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:53.044767 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.054912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d51008-83c3-43cf-bbb8-ddfd86b0803f service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.060352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.061030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.061236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.061682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.063667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:53.075096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.076558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.085902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1247/2484] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:54.572315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83d6d5-73af-41ac-b78d-a6c5da737cf1 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:54.573711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-aa83d6d5-73af-41ac-b78d-a6c5da737cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:54.645464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83d6d5-73af-41ac-b78d-a6c5da737cf1 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:54.651588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:54.651872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:54.652115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:54.652389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:54.654171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:54.665863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:54.667467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:54.677680 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1238/2485] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:03.081378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=78 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:04.673235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=50 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:29.285074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.286896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:29.376552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcde60-ba6b-4128-9417-e8ab52342302 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.380512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.381144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.381427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.381704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.384465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:29.470597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.474303 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.491423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1248/2486] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:32.535193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:32.615836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3004f-d903-47f1-b05a-f91f4474daad service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:32.624170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:32.624803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.625190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.625673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.630164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:32.729214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:32.730913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:32.746242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1239/2487] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:39.483828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=48 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:42.738187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=56 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:59.546086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:59.594437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53238d16-1576-44ba-a52f-ea4796ac7419 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:59.598475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:59.599101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:59.599390 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:59.599897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:59.602096 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:59.671025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:59.672688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:59.681949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1249/2488] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:09.677617 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=39 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:18.140982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:18.153707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa686-b743-4b26-98d9-d897c352893e service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:18.159189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:18.159794 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:18.160078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:18.160348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:18.162196 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:18.224805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:18.226419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:18.237016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1240/2489] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:28.232547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=28 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:41.996017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:42.018968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc29e4e-a80a-4bc7-8ea8-dfabccc8a586 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:42.024725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:42.025340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.025622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.025896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.027691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:42.053742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:42.055415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:42.065705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1250/2490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:52.061288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=15 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:29.981483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:30.004183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a2c1a-539e-4e29-9bc6-5b6c950cdfe0 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:30.007804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:30.008410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.008684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.008951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.019474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:30.021350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:30.033472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1241/2491] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:31.581723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:31.594799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925f40-d823-4769-bae7-b82215a665d1 service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:31.599014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35449b0f-9890-485f-8904-6d99529d2d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:31.599630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449b0f-9890-485f-8904-6d99529d2d61 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.599912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449b0f-9890-485f-8904-6d99529d2d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.600192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449b0f-9890-485f-8904-6d99529d2d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.612596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35449b0f-9890-485f-8904-6d99529d2d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:31.614649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35449b0f-9890-485f-8904-6d99529d2d61 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:31.631202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1251/2492] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:40.027543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=10 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:41.623060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:44.795462 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:44.797589 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:44.835674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6a3af-e055-46c6-9a8a-c92d808603e5 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:44.839446 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747cd29-9a3f-47df-addf-e3640a85dd5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:44.840048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cd29-9a3f-47df-addf-e3640a85dd5a service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:44.840365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cd29-9a3f-47df-addf-e3640a85dd5a service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:44.840636 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cd29-9a3f-47df-addf-e3640a85dd5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:44.852298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747cd29-9a3f-47df-addf-e3640a85dd5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:44.852591 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747cd29-9a3f-47df-addf-e3640a85dd5a service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:44.881798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1242/2493] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:47.977710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35449b0f-9890-485f-8904-6d99529d2d61 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:47.989794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35449b0f-9890-485f-8904-6d99529d2d61 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:47.995299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3ca47-9a1a-4808-bc4f-a732f903b3be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:47.996169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3ca47-9a1a-4808-bc4f-a732f903b3be service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:47.996442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3ca47-9a1a-4808-bc4f-a732f903b3be service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:47.996720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3ca47-9a1a-4808-bc4f-a732f903b3be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.005315 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3ca47-9a1a-4808-bc4f-a732f903b3be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:48.007049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3ca47-9a1a-4808-bc4f-a732f903b3be service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.037659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1252/2494] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:54.878998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=22 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:58.034552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=13 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:11.813102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747cd29-9a3f-47df-addf-e3640a85dd5a service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:11.829697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747cd29-9a3f-47df-addf-e3640a85dd5a service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:11.835700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c89fdc-f419-4092-9003-aa9d59b41fa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:11.836505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c89fdc-f419-4092-9003-aa9d59b41fa7 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:11.836924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c89fdc-f419-4092-9003-aa9d59b41fa7 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:11.837268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c89fdc-f419-4092-9003-aa9d59b41fa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:11.848417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c89fdc-f419-4092-9003-aa9d59b41fa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:11.850269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c89fdc-f419-4092-9003-aa9d59b41fa7 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:11.885621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1243/2495] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:21.881292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=13 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:41.506750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3ca47-9a1a-4808-bc4f-a732f903b3be service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:41.522429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3ca47-9a1a-4808-bc4f-a732f903b3be service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:41.527925 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4cea-e6f0-46db-8264-c054fba487f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:41.528825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4cea-e6f0-46db-8264-c054fba487f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:41.529217 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4cea-e6f0-46db-8264-c054fba487f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:41.529618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4cea-e6f0-46db-8264-c054fba487f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:41.542987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf4cea-e6f0-46db-8264-c054fba487f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:41.545170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf4cea-e6f0-46db-8264-c054fba487f5 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:41.558107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1253/2496] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:51.553166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:32.994583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c89fdc-f419-4092-9003-aa9d59b41fa7 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:33.015907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c89fdc-f419-4092-9003-aa9d59b41fa7 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:33.024048 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eef639-9087-4ba0-a25e-646b90b8c1a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:33.024746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef639-9087-4ba0-a25e-646b90b8c1a1 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:33.027767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef639-9087-4ba0-a25e-646b90b8c1a1 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:33.028023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef639-9087-4ba0-a25e-646b90b8c1a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:33.038208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eef639-9087-4ba0-a25e-646b90b8c1a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:33.039927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eef639-9087-4ba0-a25e-646b90b8c1a1 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:33.076336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1244/2497] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:43.073199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=13 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:45.199401 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4cea-e6f0-46db-8264-c054fba487f5 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:45.216974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4cea-e6f0-46db-8264-c054fba487f5 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:45.225104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:45.225823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:45.226374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:45.226732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:45.240140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:45.241853 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:45.284725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1254/2498] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:55.280932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=13 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:45.559558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eef639-9087-4ba0-a25e-646b90b8c1a1 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:45.570374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eef639-9087-4ba0-a25e-646b90b8c1a1 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:45.574375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d441f6e-b08c-417d-b295-32fbd6bb126b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:45.574989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441f6e-b08c-417d-b295-32fbd6bb126b service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:45.575263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441f6e-b08c-417d-b295-32fbd6bb126b service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:45.575526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441f6e-b08c-417d-b295-32fbd6bb126b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:45.589214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d441f6e-b08c-417d-b295-32fbd6bb126b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:45.596080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d441f6e-b08c-417d-b295-32fbd6bb126b service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:45.635712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1245/2499] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:45.943512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:45.944926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:45.960133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8e659-bf7e-40e3-9138-cfa5f02d4b29 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:45.966011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b9c62-5b2b-4ef5-8ae3-2af86547aff6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:45.967039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b9c62-5b2b-4ef5-8ae3-2af86547aff6 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:45.967039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b9c62-5b2b-4ef5-8ae3-2af86547aff6 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:45.967324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b9c62-5b2b-4ef5-8ae3-2af86547aff6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:45.977152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b9c62-5b2b-4ef5-8ae3-2af86547aff6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:45.978794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b9c62-5b2b-4ef5-8ae3-2af86547aff6 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:45.990692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1255/2500] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:55.632736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=13 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:55.985245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:31.202360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d441f6e-b08c-417d-b295-32fbd6bb126b service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:31.210848 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d441f6e-b08c-417d-b295-32fbd6bb126b service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:31.214866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:31.215468 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:31.215742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:31.216011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:31.220134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:31.321453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:31.323504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:31.334643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1246/2501] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:41.329227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=33 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:45.673623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b9c62-5b2b-4ef5-8ae3-2af86547aff6 service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:45.688531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b9c62-5b2b-4ef5-8ae3-2af86547aff6 service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:45.693120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:45.693843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:45.694348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:45.694691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:45.697170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:45.810179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:45.811638 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:45.823970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1256/2502] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:09:45.971064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:45.981351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c8a31-e3de-48bd-8d33-3a2afdd6b1ab service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:45.985321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:45.987871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:45.987871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:45.987871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:45.988753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:46.005702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:46.008082 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:46.027322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1247/2503] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:46.088125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:46.106376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e073519-80e5-4cce-9193-7f97fd5fad3b service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:46.106376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:46.106376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:46.106376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:46.108033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:46.108033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:46.263141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:46.264078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:46.264944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:46.266505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:46.279159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1257/2504] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:09:46.317556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d788f59-e20e-4554-bdb9-4fa7eb6f1f8f service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:46.317556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9633e-f319-49a1-a1a4-3b0e6c7172de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:46.317556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9633e-f319-49a1-a1a4-3b0e6c7172de service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:46.317556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9633e-f319-49a1-a1a4-3b0e6c7172de service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:46.318357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9633e-f319-49a1-a1a4-3b0e6c7172de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:46.318357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9633e-f319-49a1-a1a4-3b0e6c7172de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:46.318357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9633e-f319-49a1-a1a4-3b0e6c7172de service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:46.341359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1248/2505] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:46.483380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:46.486782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:46.606127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9bc97-8d97-4d07-a448-a64328ecaf99 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:46.611399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:46.612549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:46.612549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:46.612829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:46.615321 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:46.622759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:46.626861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:46.633451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:46.640592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1258/2506] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:46.754805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9633e-f319-49a1-a1a4-3b0e6c7172de service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:46.766862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9633e-f319-49a1-a1a4-3b0e6c7172de service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:46.774518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:46.775149 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:46.775577 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:46.776012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:46.777926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:46.873738 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:46.876528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:46.887254 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1249/2507] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:56.728842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:56.882402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=52 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:46.531647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:46.603866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fbf86-bfca-4c40-8356-f3b9691b447e service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:46.607507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:46.608138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:46.608453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:46.608764 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:46.611398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:46.696110 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:46.697648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:46.707459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1259/2508] 10.222.0.25 () {60 vars in 1396 bytes} [Thu Sep 8 14:11:46 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:11:46.934369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:46.944825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05050c18-a481-493c-ab88-e2e0992bdf4a service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:46.948459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:46.949065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:46.949349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:46.949622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:46.952763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:47.028359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:47.030740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:47.040436 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1250/2509] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:56.703198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=61 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:57.036878 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=33 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:51.936938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:52.049359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85934a-d103-4512-a024-7cecaba844de service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:52.054843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85424e-7387-4f95-9673-274569185a6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:52.055460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85424e-7387-4f95-9673-274569185a6f service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:52.055738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85424e-7387-4f95-9673-274569185a6f service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:52.056013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85424e-7387-4f95-9673-274569185a6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:52.057998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-df85424e-7387-4f95-9673-274569185a6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:52.084860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85424e-7387-4f95-9673-274569185a6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:52.086538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85424e-7387-4f95-9673-274569185a6f service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:52.096999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1260/2510] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:12:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:13:02.092427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=55 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:31.600386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:31.602382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:31.692587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c305844-cb93-42f0-b03a-c3f2d0e2957a service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:31.699089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:31.699801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:31.700297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:31.700626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:31.706321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:31.790401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:31.792324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:31.805557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1251/2511] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:32.376981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85424e-7387-4f95-9673-274569185a6f service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:32.390644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85424e-7387-4f95-9673-274569185a6f service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:32.396994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:32.397618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:32.397961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:32.398273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:32.400548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:32.414037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:32.415690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:32.428001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1261/2512] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:41.800621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=51 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:42.422326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:48.456116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:48.487849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1386b7-ead0-4a22-8610-d22f3605d174 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:48.494014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b6618-b6a3-4481-bc5b-0e2715fb99ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:48.495383 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b6618-b6a3-4481-bc5b-0e2715fb99ac service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:48.495747 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b6618-b6a3-4481-bc5b-0e2715fb99ac service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:48.496134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b6618-b6a3-4481-bc5b-0e2715fb99ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:48.506485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b6618-b6a3-4481-bc5b-0e2715fb99ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:48.507828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b6618-b6a3-4481-bc5b-0e2715fb99ac service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:48.519399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1252/2513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:14:58.513153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=10 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:31.721361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:31.723031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:31.757800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc28b23-a157-4d53-afe6-26e0037a0541 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:31.762238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca73512-a9f0-40a7-bf06-e8397602690d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:31.762824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca73512-a9f0-40a7-bf06-e8397602690d service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:31.763101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca73512-a9f0-40a7-bf06-e8397602690d service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:31.763379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca73512-a9f0-40a7-bf06-e8397602690d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:31.773895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca73512-a9f0-40a7-bf06-e8397602690d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:31.775532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca73512-a9f0-40a7-bf06-e8397602690d service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:31.787038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1262/2514] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:31.838673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b6618-b6a3-4481-bc5b-0e2715fb99ac service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:31.848308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b6618-b6a3-4481-bc5b-0e2715fb99ac service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:31.852004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb4ec-1a02-4232-b6c8-0070447f82ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:31.852614 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb4ec-1a02-4232-b6c8-0070447f82ac service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:31.852896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb4ec-1a02-4232-b6c8-0070447f82ac service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:31.853700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efb4ec-1a02-4232-b6c8-0070447f82ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:31.867863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efb4ec-1a02-4232-b6c8-0070447f82ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:31.869914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efb4ec-1a02-4232-b6c8-0070447f82ac service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:31.882990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1253/2515] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:31.932368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca73512-a9f0-40a7-bf06-e8397602690d service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:31.941047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca73512-a9f0-40a7-bf06-e8397602690d service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:31.944786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695dae9-0791-4bf0-8e20-eea236fea88f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:31.948621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695dae9-0791-4bf0-8e20-eea236fea88f service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:31.948621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695dae9-0791-4bf0-8e20-eea236fea88f service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:31.948621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695dae9-0791-4bf0-8e20-eea236fea88f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:31.955086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695dae9-0791-4bf0-8e20-eea236fea88f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:31.956551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695dae9-0791-4bf0-8e20-eea236fea88f service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:31.984580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1263/2516] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:41.878034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=6 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:41.981742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=28 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:47.314648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb4ec-1a02-4232-b6c8-0070447f82ac service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:47.323555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efb4ec-1a02-4232-b6c8-0070447f82ac service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:47.329830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c239039-94c5-4d04-9474-b6971d1ccda4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:47.330483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c239039-94c5-4d04-9474-b6971d1ccda4 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:47.330765 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c239039-94c5-4d04-9474-b6971d1ccda4 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:47.331038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c239039-94c5-4d04-9474-b6971d1ccda4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:47.340534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c239039-94c5-4d04-9474-b6971d1ccda4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:47.342366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c239039-94c5-4d04-9474-b6971d1ccda4 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:47.373051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1254/2517] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:47.376666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695dae9-0791-4bf0-8e20-eea236fea88f service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:47.389429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695dae9-0791-4bf0-8e20-eea236fea88f service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:47.395300 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:47.395912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:47.396189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:47.396456 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:47.398611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:47.483244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:47.484908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:47.497809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1264/2518] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:47.843368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c239039-94c5-4d04-9474-b6971d1ccda4 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:47.859000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c239039-94c5-4d04-9474-b6971d1ccda4 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:47.865011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:47.865631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:47.865912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:47.866214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:47.875152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:47.876722 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:47.904319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1255/2519] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:15:57.489859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=33 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:57.901186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=26 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:17:32.106539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:17:32.116845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34032c80-fce7-4ee3-92c2-a2db67bc70ae service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:17:32.120724 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cfb86-6cbd-497d-8cfb-b6dd56b20969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:17:32.121324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cfb86-6cbd-497d-8cfb-b6dd56b20969 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:17:32.121648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cfb86-6cbd-497d-8cfb-b6dd56b20969 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:17:32.121963 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cfb86-6cbd-497d-8cfb-b6dd56b20969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:17:32.131123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cfb86-6cbd-497d-8cfb-b6dd56b20969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:17:32.132558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cfb86-6cbd-497d-8cfb-b6dd56b20969 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:17:32.162899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1265/2520] 10.222.0.25 () {60 vars in 1396 bytes} [Thu Sep 8 14:17:32 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:17:42.160239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=13 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:17:57.892870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:17:57.894529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:17:57.910314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a007b-420f-4a6b-8c8b-1e605222eff2 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:17:57.916918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62837b2a-8afd-4a94-9292-a758fd79eb0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:17:57.917542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62837b2a-8afd-4a94-9292-a758fd79eb0e service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:17:57.917826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62837b2a-8afd-4a94-9292-a758fd79eb0e service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:17:57.918102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62837b2a-8afd-4a94-9292-a758fd79eb0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:17:57.927452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62837b2a-8afd-4a94-9292-a758fd79eb0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:17:57.929697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62837b2a-8afd-4a94-9292-a758fd79eb0e service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:17:57.940674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1256/2521] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:17:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:18:07.936898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=6 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:32.380229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cfb86-6cbd-497d-8cfb-b6dd56b20969 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:19:32.393139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cfb86-6cbd-497d-8cfb-b6dd56b20969 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:19:32.399377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcdfb1-f6c4-415e-b91d-cc2afa1fe096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:19:32.400066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcdfb1-f6c4-415e-b91d-cc2afa1fe096 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:19:32.400359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcdfb1-f6c4-415e-b91d-cc2afa1fe096 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:19:32.400819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcdfb1-f6c4-415e-b91d-cc2afa1fe096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:19:32.409986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcdfb1-f6c4-415e-b91d-cc2afa1fe096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:19:32.411503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcdfb1-f6c4-415e-b91d-cc2afa1fe096 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:19:32.435114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62837b2a-8afd-4a94-9292-a758fd79eb0e service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:19:32.446974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1266/2522] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:19:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:19:32.447218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62837b2a-8afd-4a94-9292-a758fd79eb0e service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:19:32.453531 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:19:32.454204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:19:32.454455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:19:32.454729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:19:32.463654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:19:32.465355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:19:32.472512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcdfb1-f6c4-415e-b91d-cc2afa1fe096 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:19:32.482711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcdfb1-f6c4-415e-b91d-cc2afa1fe096 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:19:32.486513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeed5eb-0165-4aa2-b16f-37e68ed601b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:19:32.487196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeed5eb-0165-4aa2-b16f-37e68ed601b4 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:19:32.487604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeed5eb-0165-4aa2-b16f-37e68ed601b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:19:32.487946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeed5eb-0165-4aa2-b16f-37e68ed601b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:19:32.497491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1257/2523] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:19:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:19:32.498798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeed5eb-0165-4aa2-b16f-37e68ed601b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:19:32.499716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeed5eb-0165-4aa2-b16f-37e68ed601b4 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:19:32.510509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1267/2524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:19:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:19:42.492434 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=13 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:42.506337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=19 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:54.483134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:19:54.484940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:19:54.566734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98284fff-cc3e-4d5b-8e66-d39e9b112e5e service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:19:54.572030 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:19:54.572656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:19:54.572968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:19:54.573232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:19:54.576916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:19:54.594652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:19:54.596524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:19:54.605379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1258/2525] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:19:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:20:04.601497 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=31 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:21:32.798516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeed5eb-0165-4aa2-b16f-37e68ed601b4 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:21:32.809165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeed5eb-0165-4aa2-b16f-37e68ed601b4 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:21:32.813114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:21:32.813714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:21:32.813990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:21:32.814295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:21:32.816414 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:21:32.911471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:21:32.912923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:21:32.924147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1268/2526] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:21:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:21:32.975660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:21:32.990175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbb81e-edea-4dfe-95f6-ac967ad34540 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:21:32.994052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:21:32.994815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:21:32.995163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:21:32.995562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:21:32.999132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:21:33.015875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:21:33.017745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:21:33.033361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1259/2527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:21:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:21:33.148731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:21:33.161685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d788962-226d-4dad-aa4b-3eca2ac5bbda service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:21:33.167678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:21:33.169946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:21:33.170273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:21:33.170550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:21:33.172555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:21:33.174854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:21:33.177309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:21:33.195019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f666394-0e24-44b2-b90f-597ff3d2c86a service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:21:33.204450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbd242-f787-4217-b935-f5d8da8c546b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:21:33.204450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd242-f787-4217-b935-f5d8da8c546b service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:21:33.204450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd242-f787-4217-b935-f5d8da8c546b service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:21:33.220074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd242-f787-4217-b935-f5d8da8c546b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:21:33.220074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbd242-f787-4217-b935-f5d8da8c546b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:21:33.220074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbd242-f787-4217-b935-f5d8da8c546b service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:21:33.251974 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1260/2528] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:21:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:21:33.253109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbd242-f787-4217-b935-f5d8da8c546b service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:21:33.260982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:21:33.262641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:21:33.263861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbd242-f787-4217-b935-f5d8da8c546b service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:21:33.267773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:21:33.268379 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:21:33.268676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:21:33.268938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:21:33.276429 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:21:33.276534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1269/2529] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:21:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:21:33.330849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:21:33.344386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1892b-a9dd-4983-a496-2405c674bdca service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:21:33.350247 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:21:33.351051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:21:33.351190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:21:33.351460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:21:33.353415 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:21:33.373658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:21:33.375431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:21:33.386098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1261/2530] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:21:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:21:33.427493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:21:33.439991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:21:33.441653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:21:33.452175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1270/2531] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:21:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:21:34.008914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:21:34.021417 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79498940-c3fe-4abd-a861-4819869e9bf9 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:21:34.027522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38646e4a-3688-45a5-a20f-aa191bade78e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:21:34.028116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38646e4a-3688-45a5-a20f-aa191bade78e service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:21:34.028395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38646e4a-3688-45a5-a20f-aa191bade78e service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:21:34.028673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38646e4a-3688-45a5-a20f-aa191bade78e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:21:34.037645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38646e4a-3688-45a5-a20f-aa191bade78e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:21:34.039815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38646e4a-3688-45a5-a20f-aa191bade78e service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:21:34.050833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1262/2532] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:21:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:21:43.455265 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:21:44.045898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=58 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:23:05.863591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:23:05.865155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:23:05.990726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c944b1-31fd-4398-8a5f-ee5cfdc4ea0c service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:23:05.996497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:23:05.997111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:23:05.997387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:23:05.997656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:23:05.999562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:23:06.012090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:23:06.013707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:23:06.024954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1271/2533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:23:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:23:16.019816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=52 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:23:33.608328 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38646e4a-3688-45a5-a20f-aa191bade78e service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:23:33.681127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38646e4a-3688-45a5-a20f-aa191bade78e service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:23:33.685426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:23:33.686036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:23:33.686347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:23:33.686626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:23:33.688606 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:23:33.783205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:23:33.784556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:23:33.793726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1263/2534] 10.222.0.25 () {60 vars in 1396 bytes} [Thu Sep 8 14:23:33 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:23:33.895539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:23:33.895539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:23:33.959453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f505c2-7b42-44f1-960f-d9bd0343c60f service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:23:33.967014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:23:33.967702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:23:33.967988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:23:33.968261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:23:33.970289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:23:33.986349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:23:33.987966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:23:33.999615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1272/2535] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:23:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:23:43.789964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=64 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:23:43.995023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=24 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:25:34.168132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:25:34.181809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c440b-b70a-4404-8c44-1b3a3e96d57d service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:25:34.187560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:25:34.187987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:25:34.188265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:25:34.188535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:25:34.190490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:25:34.276782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:25:34.278425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:25:34.288092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1264/2536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:25:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:25:44.283341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=33 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:25:48.200366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:25:48.202121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:25:48.243189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872ff69-59ef-4251-9342-d41699d8abd3 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:25:48.246896 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:25:48.247509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:25:48.250885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:25:48.250885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:25:48.250885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:25:48.351567 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:25:48.354429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:25:48.365448 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1273/2537] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:25:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:25:58.359932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=42 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:26:00.502874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:26:00.504493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:26:00.528057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c931bbf-3e55-469d-b3bf-9b251ac8aae1 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:26:00.531795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f816-b791-46c5-b0d3-81e4f230d547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:26:00.532403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f816-b791-46c5-b0d3-81e4f230d547 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:26:00.532678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f816-b791-46c5-b0d3-81e4f230d547 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:26:00.532946 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f816-b791-46c5-b0d3-81e4f230d547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:26:00.541136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8f816-b791-46c5-b0d3-81e4f230d547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:26:00.542664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8f816-b791-46c5-b0d3-81e4f230d547 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:26:00.551568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1265/2538] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:26:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:26:10.547407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=10 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:27:34.230608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:27:34.232229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:27:34.244242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdcf0d-6ef5-487b-93f4-a837253703f1 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:27:34.248005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ae6a3-8ef9-4c3e-8e2a-e6f9f56a942f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:27:34.248611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ae6a3-8ef9-4c3e-8e2a-e6f9f56a942f service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:27:34.248892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ae6a3-8ef9-4c3e-8e2a-e6f9f56a942f service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:27:34.249168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ae6a3-8ef9-4c3e-8e2a-e6f9f56a942f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:27:34.260438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ae6a3-8ef9-4c3e-8e2a-e6f9f56a942f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:27:34.261885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ae6a3-8ef9-4c3e-8e2a-e6f9f56a942f service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:27:34.273783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1274/2539] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:27:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:27:34.281034 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f816-b791-46c5-b0d3-81e4f230d547 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:27:34.291088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f816-b791-46c5-b0d3-81e4f230d547 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:27:34.294585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:27:34.295178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:27:34.295551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:27:34.295719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:27:34.297626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:27:34.371816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:27:34.373278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:27:34.382899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1266/2540] 10.222.0.25 () {60 vars in 1396 bytes} [Thu Sep 8 14:27:34 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:27:34.432135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ae6a3-8ef9-4c3e-8e2a-e6f9f56a942f service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:27:34.441074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ae6a3-8ef9-4c3e-8e2a-e6f9f56a942f service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:27:34.444906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d4cc7-1414-4780-9449-7182b4765fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:27:34.445521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d4cc7-1414-4780-9449-7182b4765fec service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:27:34.445801 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d4cc7-1414-4780-9449-7182b4765fec service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:27:34.446079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d4cc7-1414-4780-9449-7182b4765fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:27:34.457895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d4cc7-1414-4780-9449-7182b4765fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:27:34.459358 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d4cc7-1414-4780-9449-7182b4765fec service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:27:34.470813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1275/2541] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:27:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:27:34.519963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:27:34.528581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40df0fc-525e-4e5b-9b59-d5e23256c20d service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:27:34.532675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e710bd-2edd-4659-9afe-958714d7adf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:27:34.533286 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e710bd-2edd-4659-9afe-958714d7adf0 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:27:34.533569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e710bd-2edd-4659-9afe-958714d7adf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:27:34.533844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e710bd-2edd-4659-9afe-958714d7adf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:27:34.543955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e710bd-2edd-4659-9afe-958714d7adf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:27:34.545387 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e710bd-2edd-4659-9afe-958714d7adf0 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:27:34.579584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1267/2542] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:27:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:27:44.465703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=12 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:27:44.576640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=46 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:28:10.816476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d4cc7-1414-4780-9449-7182b4765fec service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:28:10.828074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d4cc7-1414-4780-9449-7182b4765fec service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:28:10.833310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2619-c7bb-48af-890f-6992e93e9ca0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:28:10.833954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2619-c7bb-48af-890f-6992e93e9ca0 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:28:10.834282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2619-c7bb-48af-890f-6992e93e9ca0 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:28:10.834557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2619-c7bb-48af-890f-6992e93e9ca0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:28:10.843458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d2619-c7bb-48af-890f-6992e93e9ca0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:28:10.845147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d2619-c7bb-48af-890f-6992e93e9ca0 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:28:10.856478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1276/2543] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:28:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:28:20.852613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=6 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:29:41.151585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e710bd-2edd-4659-9afe-958714d7adf0 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:41.164523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e710bd-2edd-4659-9afe-958714d7adf0 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:41.170638 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67bf104-717f-4e49-92e5-532b61112c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:41.171378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bf104-717f-4e49-92e5-532b61112c18 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:41.171727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bf104-717f-4e49-92e5-532b61112c18 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:41.172012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bf104-717f-4e49-92e5-532b61112c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:41.180773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67bf104-717f-4e49-92e5-532b61112c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:41.182464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67bf104-717f-4e49-92e5-532b61112c18 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:41.211967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1268/2544] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:41.258044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2619-c7bb-48af-890f-6992e93e9ca0 service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:41.271870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2619-c7bb-48af-890f-6992e93e9ca0 service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:41.277713 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c3d2-1696-4691-a7e2-ce0c3d94d196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:41.278306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c3d2-1696-4691-a7e2-ce0c3d94d196 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:41.278582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c3d2-1696-4691-a7e2-ce0c3d94d196 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:41.278856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c3d2-1696-4691-a7e2-ce0c3d94d196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:41.288163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6c3d2-1696-4691-a7e2-ce0c3d94d196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:41.291457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6c3d2-1696-4691-a7e2-ce0c3d94d196 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:41.347534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1277/2545] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:41.584212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67bf104-717f-4e49-92e5-532b61112c18 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:41.604544 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67bf104-717f-4e49-92e5-532b61112c18 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:41.611263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae3113-2a03-4379-82f6-548a8131245b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:41.612854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae3113-2a03-4379-82f6-548a8131245b service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:41.612854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae3113-2a03-4379-82f6-548a8131245b service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:41.612854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae3113-2a03-4379-82f6-548a8131245b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:41.622674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae3113-2a03-4379-82f6-548a8131245b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:41.624595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae3113-2a03-4379-82f6-548a8131245b service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:41.636920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1269/2546] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:42.102599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c3d2-1696-4691-a7e2-ce0c3d94d196 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:42.112144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c3d2-1696-4691-a7e2-ce0c3d94d196 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:42.116732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fd558-8f33-4ba0-ae0a-1c01c71f9a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:42.117416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fd558-8f33-4ba0-ae0a-1c01c71f9a33 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:42.117769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fd558-8f33-4ba0-ae0a-1c01c71f9a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:42.118166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fd558-8f33-4ba0-ae0a-1c01c71f9a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:42.128181 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fd558-8f33-4ba0-ae0a-1c01c71f9a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:42.129937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fd558-8f33-4ba0-ae0a-1c01c71f9a33 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:42.167915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1278/2547] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:43.600660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae3113-2a03-4379-82f6-548a8131245b service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:43.615247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae3113-2a03-4379-82f6-548a8131245b service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:43.619143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:43.628708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:43.628708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:43.628708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:43.628708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:43.754517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:43.758115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:43.785152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1270/2548] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:45.539686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fd558-8f33-4ba0-ae0a-1c01c71f9a33 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:45.550442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fd558-8f33-4ba0-ae0a-1c01c71f9a33 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:45.554204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c817510-81ce-4305-941a-d58e61815c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:45.554811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c817510-81ce-4305-941a-d58e61815c5b service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:45.555092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c817510-81ce-4305-941a-d58e61815c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:45.555363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c817510-81ce-4305-941a-d58e61815c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:45.557272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8c817510-81ce-4305-941a-d58e61815c5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:45.577309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c817510-81ce-4305-941a-d58e61815c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:45.578921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c817510-81ce-4305-941a-d58e61815c5b service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:45.592694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1279/2549] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:45.958524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:45.960192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09acbe-b946-40e4-aa1d-42be4cef9de8 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:45.966014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acd1ae-594e-4438-bc0c-488785421dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:45.966684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acd1ae-594e-4438-bc0c-488785421dbc service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:45.966967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acd1ae-594e-4438-bc0c-488785421dbc service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:45.967243 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acd1ae-594e-4438-bc0c-488785421dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:45.969152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-78acd1ae-594e-4438-bc0c-488785421dbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:45.985934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78acd1ae-594e-4438-bc0c-488785421dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:45.987799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78acd1ae-594e-4438-bc0c-488785421dbc service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:46.000395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1271/2550] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:29:49.540486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b3f84-e14f-438d-9599-b9388b7ee5de None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:49.541030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b3f84-e14f-438d-9599-b9388b7ee5de None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:49.541030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b3f84-e14f-438d-9599-b9388b7ee5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:49.577568 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-114b3f84-e14f-438d-9599-b9388b7ee5de None None] MFA Rules not processed for user `eca89f435299415c8448931e73f5b0fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:29:49.579753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-114b3f84-e14f-438d-9599-b9388b7ee5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:49.589030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1280/2551] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:29:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:29:49.609704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcd6f9-51ec-41c7-a284-41121d7622b7 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:49.610190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcd6f9-51ec-41c7-a284-41121d7622b7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:49.610190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcd6f9-51ec-41c7-a284-41121d7622b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:49.770164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ecfcd6f9-51ec-41c7-a284-41121d7622b7 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:29:49.773247 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ecfcd6f9-51ec-41c7-a284-41121d7622b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:49.841356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:29:49.921987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1272/2552] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 14:29:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:29:50.067474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b3f84-e14f-438d-9599-b9388b7ee5de None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:50.069536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-114b3f84-e14f-438d-9599-b9388b7ee5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:50.121800 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc11a1d-e8c9-44de-835e-6f8ff02236bf None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:50.122070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc11a1d-e8c9-44de-835e-6f8ff02236bf None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:50.122469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc11a1d-e8c9-44de-835e-6f8ff02236bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:50.181201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6dc11a1d-e8c9-44de-835e-6f8ff02236bf None None] MFA Rules not processed for user `e9925313d1d446fa81a00cb55348dff7`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:29:50.186444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6dc11a1d-e8c9-44de-835e-6f8ff02236bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:50.208790 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1273/2553] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:29:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:29:50.224174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc11a1d-e8c9-44de-835e-6f8ff02236bf None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:50.236908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc11a1d-e8c9-44de-835e-6f8ff02236bf None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:50.241872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:50.244661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b3f84-e14f-438d-9599-b9388b7ee5de None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:50.259336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:50.259586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ad204-c965-4f5a-a3d9-1b363af42375 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:50.260082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad204-c965-4f5a-a3d9-1b363af42375 service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:50.268787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad204-c965-4f5a-a3d9-1b363af42375 service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:50.268787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad204-c965-4f5a-a3d9-1b363af42375 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:50.269065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:50.269065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:50.269065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:50.276768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ad204-c965-4f5a-a3d9-1b363af42375 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:50.281821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ad204-c965-4f5a-a3d9-1b363af42375 service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:50.294773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:50.298561 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:50.315032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1281/2554] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:29:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:50.333045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1274/2555] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:29:50.881038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ad204-c965-4f5a-a3d9-1b363af42375 service swift] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:50.899242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ad204-c965-4f5a-a3d9-1b363af42375 service swift] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:50.905852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:50.907197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:50.907826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:50.908357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:50.910857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:50.924996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:50.926698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:50.975589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1282/2556] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:51.283176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:51.295920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea9b02-2de6-405e-8799-2d7a7c72942b service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:51.304016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:51.304888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:51.305231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:51.305518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:51.310955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:51.333256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:51.335288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:51.345124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1275/2557] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:51.623857 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:51.625548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:51.648092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b356ab-57a8-4654-8045-ee7828101738 service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:51.657838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae080d72-5a8a-4c3c-84f1-6ed961d5d179 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:51.662445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae080d72-5a8a-4c3c-84f1-6ed961d5d179 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:51.662445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae080d72-5a8a-4c3c-84f1-6ed961d5d179 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:51.662445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae080d72-5a8a-4c3c-84f1-6ed961d5d179 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:51.662445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ae080d72-5a8a-4c3c-84f1-6ed961d5d179 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:51.688143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae080d72-5a8a-4c3c-84f1-6ed961d5d179 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:51.692198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae080d72-5a8a-4c3c-84f1-6ed961d5d179 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:51.714554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1283/2558] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:51.848235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:29:51.861782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036446-8c42-457b-8ccc-d3a1c3140f46 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:29:51.872832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:29:51.873798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:51.874273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:51.874727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:51.879773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:51.898670 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:29:51.900181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:29:51.915024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1276/2559] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:29:59.187777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af694e-176c-48be-a1df-b012a0bd2f70 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:29:59.188805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af694e-176c-48be-a1df-b012a0bd2f70 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:29:59.189253 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af694e-176c-48be-a1df-b012a0bd2f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:29:59.235542 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-d8af694e-176c-48be-a1df-b012a0bd2f70 None None] MFA Rules not processed for user `5ae9377cf36e4086aa38c07b021fdead`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:29:59.237029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d8af694e-176c-48be-a1df-b012a0bd2f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:29:59.247251 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1284/2560] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:29:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:30:01.203449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:01.220964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d64e5b-1c64-444f-9269-a35229a3c4c0 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:01.248753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:01.249399 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:01.249676 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:01.249950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:01.256593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:01.291981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:01.293649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:01.315440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1277/2561] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:30:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:30:01.775042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af694e-176c-48be-a1df-b012a0bd2f70 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:01.778359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d8af694e-176c-48be-a1df-b012a0bd2f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:01.806156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af694e-176c-48be-a1df-b012a0bd2f70 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:01.815724 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:01.825782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:01.825782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:01.825782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] PATH_INFO: `/v3/users/772205d93ce047e9ad4ddefadec300f4` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:01.825782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=772205d93ce047e9ad4ddefadec300f4)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:01.825782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:01.825782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:02.033890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '772205d93ce047e9ad4ddefadec300f4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:02.092140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '772205d93ce047e9ad4ddefadec300f4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:02.092953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '772205d93ce047e9ad4ddefadec300f4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:02.111460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '772205d93ce047e9ad4ddefadec300f4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:02.121677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '772205d93ce047e9ad4ddefadec300f4'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:02.128462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1285/2562] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:30:01 2022] DELETE /identity/v3/users/772205d93ce047e9ad4ddefadec300f4 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:02.702871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:02.704486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:02.724388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1048e-6913-4edb-8d55-a170d6799af9 service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:02.823867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:02.825319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:02.828216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:02.832408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:02.839989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:02.840570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] PATH_INFO: `/v3/users/f2026170f4674b63a37534c14689bd10` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:02.847494 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2026170f4674b63a37534c14689bd10)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:02.851101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:02.885618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3293ba-0a42-4e93-ac48-377137cdcd46 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:02.911593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:02.912425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:02.912706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:02.913083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] PATH_INFO: `/v3/projects/45579d93010d4ed0b80cad3a8b85ea50` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:02.947804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45579d93010d4ed0b80cad3a8b85ea50)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:02.950405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:03.117313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45579d93010d4ed0b80cad3a8b85ea50', 'request_id': 'req-63d83f39-91ad-4fab-8585-8bdabda8b920'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.118424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2026170f4674b63a37534c14689bd10'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.130123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45579d93010d4ed0b80cad3a8b85ea50', 'request_id': 'req-63d83f39-91ad-4fab-8585-8bdabda8b920'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.226953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2026170f4674b63a37534c14689bd10'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.232481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2026170f4674b63a37534c14689bd10'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.234701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2026170f4674b63a37534c14689bd10'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.246746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2026170f4674b63a37534c14689bd10'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.260349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:03.274716 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1278/2563] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:30:02 2022] DELETE /identity/v3/users/f2026170f4674b63a37534c14689bd10 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:03.349144 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45579d93010d4ed0b80cad3a8b85ea50', 'request_id': 'req-63d83f39-91ad-4fab-8585-8bdabda8b920'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.365475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] The token cache is being invalidate because project 45579d93010d4ed0b80cad3a8b85ea50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:30:03.366164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.367058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:03.369339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1286/2564] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:30:02 2022] DELETE /identity/v3/projects/45579d93010d4ed0b80cad3a8b85ea50 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:03.373956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:03.375787 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:03.485199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281d57-8c74-47ed-a64d-21c3f32ec1db admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:03.493490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:03.495453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:03.495453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:03.495453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] PATH_INFO: `/v3/users/e41a2cf9106b444ea98ec068f79b40db` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:03.533166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41a2cf9106b444ea98ec068f79b40db)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:03.536648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:03.873437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:03.874961 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:03.930618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41a2cf9106b444ea98ec068f79b40db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.088539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:04.103578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41a2cf9106b444ea98ec068f79b40db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.104526 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41a2cf9106b444ea98ec068f79b40db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.105363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41a2cf9106b444ea98ec068f79b40db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.110354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d83f39-91ad-4fab-8585-8bdabda8b920 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:04.120729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41a2cf9106b444ea98ec068f79b40db'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.120861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:04.120861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:04.121061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:04.122248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] PATH_INFO: `/v3/projects/5be47c3b2c4c4fc2a1db117433cbcfc4` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:04.142975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5be47c3b2c4c4fc2a1db117433cbcfc4)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:04.146319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:04.147862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1279/2565] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:30:03 2022] DELETE /identity/v3/users/e41a2cf9106b444ea98ec068f79b40db => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:04.309858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'request_id': 'req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.336816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'request_id': 'req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.434623 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'request_id': 'req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.454812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] The token cache is being invalidate because project 5be47c3b2c4c4fc2a1db117433cbcfc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:30:04.456836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.458427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.461644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1287/2566] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:30:03 2022] DELETE /identity/v3/projects/5be47c3b2c4c4fc2a1db117433cbcfc4 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:04.466368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:04.467864 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:04.541874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40691-2231-4e6a-a9b2-266822d9de16 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:04.631427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:04.632323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:04.632696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:04.632978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] PATH_INFO: `/v3/users/b153d726cb7b4193a7cf5516306fa6fd` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:04.662229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b153d726cb7b4193a7cf5516306fa6fd)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:04.664967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:04.701855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:04.703472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:04.723471 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd61d17-9e4d-4fe8-b59c-a30b552f17ba admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:04.729901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:04.750326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:04.750326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:04.750326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] PATH_INFO: `/v3/projects/258b970d576d4469bf8592de0f6eaf57` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:04.750326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=258b970d576d4469bf8592de0f6eaf57)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:04.750326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:04.881883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b153d726cb7b4193a7cf5516306fa6fd'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.899994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258b970d576d4469bf8592de0f6eaf57', 'request_id': 'req-37e79010-996a-4e47-8943-404a092263aa'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.910083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258b970d576d4469bf8592de0f6eaf57', 'request_id': 'req-37e79010-996a-4e47-8943-404a092263aa'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.994197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b153d726cb7b4193a7cf5516306fa6fd'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.995010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b153d726cb7b4193a7cf5516306fa6fd'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:04.995697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b153d726cb7b4193a7cf5516306fa6fd'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:05.010498 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b153d726cb7b4193a7cf5516306fa6fd'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:05.041155 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1280/2567] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:30:04 2022] DELETE /identity/v3/users/b153d726cb7b4193a7cf5516306fa6fd => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:05.051767 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258b970d576d4469bf8592de0f6eaf57', 'request_id': 'req-37e79010-996a-4e47-8943-404a092263aa'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:05.067775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] The token cache is being invalidate because project 258b970d576d4469bf8592de0f6eaf57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:30:05.068437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:05.069322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:05.088129 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1288/2568] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:30:04 2022] DELETE /identity/v3/projects/258b970d576d4469bf8592de0f6eaf57 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:05.923307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:05.931012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:05.932735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:06.034062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16562d-a935-44f3-969a-c9fa1bbbc910 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:06.039473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:06.040197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:06.040573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:06.040923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] PATH_INFO: `/v3/projects/3f67c7acd4c9459c9e864526acd53eca` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:06.052551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f67c7acd4c9459c9e864526acd53eca)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:06.054523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:06.228652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f67c7acd4c9459c9e864526acd53eca', 'request_id': 'req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:06.254376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f67c7acd4c9459c9e864526acd53eca', 'request_id': 'req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:06.354690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f67c7acd4c9459c9e864526acd53eca', 'request_id': 'req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:06.380948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] The token cache is being invalidate because project 3f67c7acd4c9459c9e864526acd53eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:30:06.381248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:06.381933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:06.383456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1281/2569] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:30:05 2022] DELETE /identity/v3/projects/3f67c7acd4c9459c9e864526acd53eca => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:30:06.548155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:06.550389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:06.729690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:06.791830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e79010-996a-4e47-8943-404a092263aa admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:06.938001 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:06.940228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:06.941078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:06.941582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:06.958941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:06.995241 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:06.997179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:07.030728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1289/2570] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:30:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:30:07.684988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:07.686503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:07.708527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:07.798697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6641cc-5d1a-4dbb-94eb-1fd4b1af67ce admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:07.798697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:07.798697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:07.798697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:07.799508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:07.799508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:07.904705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:07.906217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:07.921416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1282/2571] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:30:09.237120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:09.238710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:09.255042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dea237-5d62-40b4-85f2-7ff72aad6c5c service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:09.261097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:09.262294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:09.263049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:09.263424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] PATH_INFO: `/v3/users/1f10a95e49744a6dbb73e54e00d81add` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:09.264615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f10a95e49744a6dbb73e54e00d81add)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:09.266725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:09.481491 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f10a95e49744a6dbb73e54e00d81add'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:09.554554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f10a95e49744a6dbb73e54e00d81add'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:09.555222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f10a95e49744a6dbb73e54e00d81add'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:09.583618 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f10a95e49744a6dbb73e54e00d81add'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:09.608735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f10a95e49744a6dbb73e54e00d81add'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:09.618462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1290/2572] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:30:09 2022] DELETE /identity/v3/users/1f10a95e49744a6dbb73e54e00d81add => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:09.933365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:09.948354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624547f-36c4-4def-8f3a-273da6732433 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:09.998360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:10.032050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:10.032731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:10.033010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:10.033398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] PATH_INFO: `/v3/projects/179292c42f8e4041adc935c80dba0f70` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:10.034626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] RBAC: Authorizing `identity:delete_project(project_id=179292c42f8e4041adc935c80dba0f70)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:10.036569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:10.199020 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179292c42f8e4041adc935c80dba0f70', 'request_id': 'req-e7677303-0304-457e-b688-6b3fdffd430f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.215585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179292c42f8e4041adc935c80dba0f70', 'request_id': 'req-e7677303-0304-457e-b688-6b3fdffd430f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.349276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179292c42f8e4041adc935c80dba0f70', 'request_id': 'req-e7677303-0304-457e-b688-6b3fdffd430f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.378131 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] The token cache is being invalidate because project 179292c42f8e4041adc935c80dba0f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:30:10.378923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.379896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.382767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1283/2573] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:30:09 2022] DELETE /identity/v3/projects/179292c42f8e4041adc935c80dba0f70 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:10.387660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:10.394374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:10.405786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b0a3-078e-4dad-97da-5cf7afb81b33 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:10.464920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:10.481972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:10.482661 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:10.482947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:10.483288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] PATH_INFO: `/v3/users/21e143acdc0d415b8575c22c7a320572` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:10.511381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e143acdc0d415b8575c22c7a320572)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:10.515240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:10.723454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e143acdc0d415b8575c22c7a320572'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.788058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e143acdc0d415b8575c22c7a320572'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.788687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e143acdc0d415b8575c22c7a320572'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.803303 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e143acdc0d415b8575c22c7a320572'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.811477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e143acdc0d415b8575c22c7a320572'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:10.817964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1291/2574] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:30:10 2022] DELETE /identity/v3/users/21e143acdc0d415b8575c22c7a320572 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:11.123145 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:11.125584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:11.139664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7677303-0304-457e-b688-6b3fdffd430f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:11.238835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:11.240111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:11.240684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:11.241158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] PATH_INFO: `/v3/projects/f8ded52ce16d4292983b3e8159d63c03` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:11.260004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ded52ce16d4292983b3e8159d63c03)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:11.262940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:11.421725 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ded52ce16d4292983b3e8159d63c03', 'request_id': 'req-38ce313e-af6c-4c22-be84-3c865fc15932'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:11.434314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ded52ce16d4292983b3e8159d63c03', 'request_id': 'req-38ce313e-af6c-4c22-be84-3c865fc15932'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:11.437114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:11.525936 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ded52ce16d4292983b3e8159d63c03', 'request_id': 'req-38ce313e-af6c-4c22-be84-3c865fc15932'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:11.555459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] The token cache is being invalidate because project f8ded52ce16d4292983b3e8159d63c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:30:11.556312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:11.557771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:11.560177 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1284/2575] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:30:11 2022] DELETE /identity/v3/projects/f8ded52ce16d4292983b3e8159d63c03 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:14.933161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:14.935254 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:14.959034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d10b2-ee0d-42dd-8ac9-f604d666b3a1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:15.100336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:15.100987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:15.101268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:15.101537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] PATH_INFO: `/v3/users/792808dcbf894aa98c671cf43ac21b8b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:15.103062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=792808dcbf894aa98c671cf43ac21b8b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:15.106539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:15.367660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792808dcbf894aa98c671cf43ac21b8b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:15.417173 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792808dcbf894aa98c671cf43ac21b8b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:15.417887 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792808dcbf894aa98c671cf43ac21b8b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:15.419888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:15.441879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792808dcbf894aa98c671cf43ac21b8b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:15.457935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-53081ded-32ea-42b1-a999-088ca5446599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792808dcbf894aa98c671cf43ac21b8b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:15.470693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1292/2576] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:30:14 2022] DELETE /identity/v3/users/792808dcbf894aa98c671cf43ac21b8b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:15.758194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:15.759914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:15.778943 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce313e-af6c-4c22-be84-3c865fc15932 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:15.862788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:15.863083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:15.863206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:15.863488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] PATH_INFO: `/v3/projects/1346d4105dce4c6e928fcb11cde56854` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:15.865095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1346d4105dce4c6e928fcb11cde56854)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:15.867148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:15.991296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1346d4105dce4c6e928fcb11cde56854', 'request_id': 'req-439c6c7d-1e23-4168-a940-8a9a389a920d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:16.000987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1346d4105dce4c6e928fcb11cde56854', 'request_id': 'req-439c6c7d-1e23-4168-a940-8a9a389a920d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:16.074547 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1346d4105dce4c6e928fcb11cde56854', 'request_id': 'req-439c6c7d-1e23-4168-a940-8a9a389a920d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:16.090499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] The token cache is being invalidate because project 1346d4105dce4c6e928fcb11cde56854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:30:16.090799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:16.091577 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-439c6c7d-1e23-4168-a940-8a9a389a920d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:30:16.097514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1285/2577] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:30:15 2022] DELETE /identity/v3/projects/1346d4105dce4c6e928fcb11cde56854 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:16.098537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b6feb-73cb-4bef-a9a6-85df75b18d8c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.099014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b6feb-73cb-4bef-a9a6-85df75b18d8c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.099128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b6feb-73cb-4bef-a9a6-85df75b18d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.164476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-452b6feb-73cb-4bef-a9a6-85df75b18d8c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:30:16.166630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-452b6feb-73cb-4bef-a9a6-85df75b18d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:16.257878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1293/2578] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:30:16.263926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d186-cc99-4c4b-aad7-edd38e0c187d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.264221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d186-cc99-4c4b-aad7-edd38e0c187d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.264500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d186-cc99-4c4b-aad7-edd38e0c187d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.286458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-55b8d186-cc99-4c4b-aad7-edd38e0c187d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:30:16.287721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-55b8d186-cc99-4c4b-aad7-edd38e0c187d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:16.302300 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1286/2579] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:30:16.303315 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b6feb-73cb-4bef-a9a6-85df75b18d8c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:16.314376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b6feb-73cb-4bef-a9a6-85df75b18d8c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:16.323113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:16.324824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.325113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.325391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.326675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:30:16.326994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:16.328772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:16.351192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1294/2580] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:30:16.354088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d186-cc99-4c4b-aad7-edd38e0c187d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:16.368259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d186-cc99-4c4b-aad7-edd38e0c187d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:16.379799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55871000-041b-4d5f-840c-e5576e10b67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:16.380427 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55871000-041b-4d5f-840c-e5576e10b67c admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.380708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55871000-041b-4d5f-840c-e5576e10b67c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.380981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55871000-041b-4d5f-840c-e5576e10b67c admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.381619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55871000-041b-4d5f-840c-e5576e10b67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:16.383792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55871000-041b-4d5f-840c-e5576e10b67c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:16.492809 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1287/2581] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:30:16 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:30:16.498161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:16.508946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f8f3-371b-4e12-87e9-13469e5f9d65 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:16.575595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389ea0d-5369-43be-bdbc-6113f67cb171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:16.576345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389ea0d-5369-43be-bdbc-6113f67cb171 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.576691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389ea0d-5369-43be-bdbc-6113f67cb171 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.576976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389ea0d-5369-43be-bdbc-6113f67cb171 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.577625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389ea0d-5369-43be-bdbc-6113f67cb171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:16.579604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389ea0d-5369-43be-bdbc-6113f67cb171 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:16.650998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1295/2582] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:30:16 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:30:16.653527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55871000-041b-4d5f-840c-e5576e10b67c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:16.664660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55871000-041b-4d5f-840c-e5576e10b67c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:16.670528 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6d4c0-bb7e-4cbb-8d1d-62df71813f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:16.671154 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d4c0-bb7e-4cbb-8d1d-62df71813f6a admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.671438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d4c0-bb7e-4cbb-8d1d-62df71813f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.671741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d4c0-bb7e-4cbb-8d1d-62df71813f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.672518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d4c0-bb7e-4cbb-8d1d-62df71813f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:16.674336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d4c0-bb7e-4cbb-8d1d-62df71813f6a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:16.686381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1288/2583] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:30:16.690707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389ea0d-5369-43be-bdbc-6113f67cb171 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:16.702445 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389ea0d-5369-43be-bdbc-6113f67cb171 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:16.707140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638c60d-37bb-4796-9bd6-c0b53f149e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:16.707986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638c60d-37bb-4796-9bd6-c0b53f149e9f admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.707986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638c60d-37bb-4796-9bd6-c0b53f149e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.708263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638c60d-37bb-4796-9bd6-c0b53f149e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.708898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638c60d-37bb-4796-9bd6-c0b53f149e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:16.710613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638c60d-37bb-4796-9bd6-c0b53f149e9f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:16.723234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1296/2584] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:30:16.727439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6d4c0-bb7e-4cbb-8d1d-62df71813f6a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:16.738740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6d4c0-bb7e-4cbb-8d1d-62df71813f6a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:16.743784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:16.744463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.744748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.745021 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] PATH_INFO: `/v3/projects/b21f666643b74b5cba0b7597ae1aa0cd/users/357ebb86fd664e08b15bf440e2a61560/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.764092 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21f666643b74b5cba0b7597ae1aa0cd, user_id=357ebb86fd664e08b15bf440e2a61560, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:16.766804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:16.836226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:16.842487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1289/2585] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:30:16 2022] PUT /identity/v3/projects/b21f666643b74b5cba0b7597ae1aa0cd/users/357ebb86fd664e08b15bf440e2a61560/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:30:16.849287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60fb1-44b6-4b56-874a-4fda9bc1287c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:16.849287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60fb1-44b6-4b56-874a-4fda9bc1287c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:16.849743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60fb1-44b6-4b56-874a-4fda9bc1287c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:16.867650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:16.873923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-77c60fb1-44b6-4b56-874a-4fda9bc1287c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:30:16.875310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-77c60fb1-44b6-4b56-874a-4fda9bc1287c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:16.940682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1297/2586] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:30:16.950697 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:16.952523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:17.011970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf5ad2-7e49-4780-a6fb-6f68204f573d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:17.015934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:17.016548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:17.016828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:17.017102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:17.026820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:17.029000 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:17.039956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1290/2587] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:30:22.788041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a24c76-d3fe-4999-8856-6d1f5c55ab09 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:22.788555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a24c76-d3fe-4999-8856-6d1f5c55ab09 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:22.788555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a24c76-d3fe-4999-8856-6d1f5c55ab09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:22.820319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-27a24c76-d3fe-4999-8856-6d1f5c55ab09 None None] MFA Rules not processed for user `357ebb86fd664e08b15bf440e2a61560`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:30:22.821666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-27a24c76-d3fe-4999-8856-6d1f5c55ab09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:22.913909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1298/2588] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:30:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:30:22.923995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:30:22.931894 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:23.026060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23f84-d28e-48e3-94eb-b55b94dc1f19 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:30:23.031904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4e6cb-214a-406c-8170-be9ffdca0dd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:30:23.032523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4e6cb-214a-406c-8170-be9ffdca0dd7 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:23.032805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4e6cb-214a-406c-8170-be9ffdca0dd7 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:23.033081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4e6cb-214a-406c-8170-be9ffdca0dd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:23.042081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4e6cb-214a-406c-8170-be9ffdca0dd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:30:23.043960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4e6cb-214a-406c-8170-be9ffdca0dd7 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:30:23.054129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1291/2589] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:30:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:30:25.525827 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846939-3f46-4818-a05c-c4e9528affc9 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:30:25.526424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846939-3f46-4818-a05c-c4e9528affc9 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:30:25.526424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846939-3f46-4818-a05c-c4e9528affc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:30:25.546339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-85846939-3f46-4818-a05c-c4e9528affc9 None None] MFA Rules not processed for user `357ebb86fd664e08b15bf440e2a61560`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:30:25.547669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-85846939-3f46-4818-a05c-c4e9528affc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:30:25.554874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1299/2590] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:30:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:30:33.049355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=60 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:35.543447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=59 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:04.454578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4e6cb-214a-406c-8170-be9ffdca0dd7 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:04.464499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4e6cb-214a-406c-8170-be9ffdca0dd7 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:04.468542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:04.469155 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:04.469433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:04.469704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:04.471503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:04.481775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:04.483265 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:04.491933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1292/2591] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:31:14.490907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=6 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:18.903693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846939-3f46-4818-a05c-c4e9528affc9 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:18.905136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-85846939-3f46-4818-a05c-c4e9528affc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:18.917708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846939-3f46-4818-a05c-c4e9528affc9 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:18.921313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:18.921923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:18.922232 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:18.922512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:18.924876 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:18.937335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:18.938839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:18.948908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1300/2592] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:19.890716 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:19.892402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:19.914358 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3688b7-5966-447a-a464-4c65a909ee67 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:19.919461 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:19.920171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:19.920516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:19.920854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] PATH_INFO: `/v3/users/5ae9377cf36e4086aa38c07b021fdead` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:19.922129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae9377cf36e4086aa38c07b021fdead)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:19.924837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:20.132706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae9377cf36e4086aa38c07b021fdead'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.210721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae9377cf36e4086aa38c07b021fdead'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.211440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae9377cf36e4086aa38c07b021fdead'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.214100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae9377cf36e4086aa38c07b021fdead'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.226309 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae9377cf36e4086aa38c07b021fdead'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.248343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1293/2593] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:31:19 2022] DELETE /identity/v3/users/5ae9377cf36e4086aa38c07b021fdead => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:20.527575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:20.539049 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3ce4a-5f05-45e3-88ba-4ef9ae4d0388 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:20.610647 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:20.611348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:20.611994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:20.612467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] PATH_INFO: `/v3/projects/1363660dde8f4e4295adc44c6ab87f8b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:20.613686 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1363660dde8f4e4295adc44c6ab87f8b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:20.615758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:20.753222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1363660dde8f4e4295adc44c6ab87f8b', 'request_id': 'req-8d228212-b018-49a7-bddc-fcf810441b3a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.771728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1363660dde8f4e4295adc44c6ab87f8b', 'request_id': 'req-8d228212-b018-49a7-bddc-fcf810441b3a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.891035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1363660dde8f4e4295adc44c6ab87f8b', 'request_id': 'req-8d228212-b018-49a7-bddc-fcf810441b3a'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.903921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] The token cache is being invalidate because project 1363660dde8f4e4295adc44c6ab87f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:31:20.904211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.926323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-8d228212-b018-49a7-bddc-fcf810441b3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:20.926323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1301/2594] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:31:20 2022] DELETE /identity/v3/projects/1363660dde8f4e4295adc44c6ab87f8b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:26.894111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:26.895566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:26.955023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f6e3-2c8c-4ffd-954b-85e2913e713b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:27.006169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:27.006818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:27.007101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:27.007372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:27.009140 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:27.054588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:27.075677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:27.077140 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:27.086693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1294/2595] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:28.113742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af5c23-3041-4bec-b932-29870f4ebeb3 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:28.113742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af5c23-3041-4bec-b932-29870f4ebeb3 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:28.113742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af5c23-3041-4bec-b932-29870f4ebeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:28.144999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c9af5c23-3041-4bec-b932-29870f4ebeb3 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:28.146549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c9af5c23-3041-4bec-b932-29870f4ebeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:28.250013 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1302/2596] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 14:31:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:31:28.399437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:28.401041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:28.508663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e39548-ef73-4446-989e-607bd282f172 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:28.514500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9722f-1856-43ff-b134-5712a0367010 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:28.520801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9722f-1856-43ff-b134-5712a0367010 service swift] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:28.520801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9722f-1856-43ff-b134-5712a0367010 service swift] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:28.520801 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9722f-1856-43ff-b134-5712a0367010 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:28.526245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9722f-1856-43ff-b134-5712a0367010 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:28.527902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9722f-1856-43ff-b134-5712a0367010 service swift] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:28.541768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1295/2597] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:31:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:31:32.249497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af5c23-3041-4bec-b932-29870f4ebeb3 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:32.249497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c9af5c23-3041-4bec-b932-29870f4ebeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:32.268660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af5c23-3041-4bec-b932-29870f4ebeb3 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:32.285712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:32.285712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:32.285712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:32.285712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] PATH_INFO: `/v3/users/eca89f435299415c8448931e73f5b0fc` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:32.286444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca89f435299415c8448931e73f5b0fc)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:32.286444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:32.295354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:32.480837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca89f435299415c8448931e73f5b0fc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:32.542332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca89f435299415c8448931e73f5b0fc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:32.543268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca89f435299415c8448931e73f5b0fc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:32.558666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca89f435299415c8448931e73f5b0fc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:32.566317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-74c6378e-b634-4e3d-ad7b-ae11a0cc18b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca89f435299415c8448931e73f5b0fc'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:32.574051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1303/2598] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:31:32 2022] DELETE /identity/v3/users/eca89f435299415c8448931e73f5b0fc => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:32.852250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9722f-1856-43ff-b134-5712a0367010 service swift] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:32.868545 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9722f-1856-43ff-b134-5712a0367010 service swift] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:32.967816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:32.968746 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:32.969523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:32.969804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] PATH_INFO: `/v3/projects/a4e0fba6cab34c429c73fcfac7ebfd94` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:32.971214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e0fba6cab34c429c73fcfac7ebfd94)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:32.973844 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:33.086400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'request_id': 'req-d2220de2-e84e-44fe-978f-5f906e7313eb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:33.094804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'request_id': 'req-d2220de2-e84e-44fe-978f-5f906e7313eb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:33.153737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'request_id': 'req-d2220de2-e84e-44fe-978f-5f906e7313eb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:33.166567 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:33.173448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] The token cache is being invalidate because project a4e0fba6cab34c429c73fcfac7ebfd94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:31:33.173448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:33.174281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d2220de2-e84e-44fe-978f-5f906e7313eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:33.176159 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1296/2599] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:31:32 2022] DELETE /identity/v3/projects/a4e0fba6cab34c429c73fcfac7ebfd94 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:33.181791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebe78d-213f-4196-be24-e6cbff6a5b5d None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:33.182378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebe78d-213f-4196-be24-e6cbff6a5b5d None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:33.182911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebe78d-213f-4196-be24-e6cbff6a5b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:33.213444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c9ebe78d-213f-4196-be24-e6cbff6a5b5d None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:33.214793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c9ebe78d-213f-4196-be24-e6cbff6a5b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:33.348927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1304/2600] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:33.356920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4d6b4-0eaa-480c-ae96-e5a7a19882e4 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:33.357349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4d6b4-0eaa-480c-ae96-e5a7a19882e4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:33.357536 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4d6b4-0eaa-480c-ae96-e5a7a19882e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:33.392415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-30f4d6b4-0eaa-480c-ae96-e5a7a19882e4 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:33.395202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-30f4d6b4-0eaa-480c-ae96-e5a7a19882e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:33.433389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1297/2601] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:33.435712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebe78d-213f-4196-be24-e6cbff6a5b5d None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:33.452579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebe78d-213f-4196-be24-e6cbff6a5b5d None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:33.467123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:33.467914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:33.470382 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:33.470729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:33.472037 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:31:33.474489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:33.476543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:33.491452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1305/2602] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:31:33.506655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4d6b4-0eaa-480c-ae96-e5a7a19882e4 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:33.517314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4d6b4-0eaa-480c-ae96-e5a7a19882e4 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:33.523535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422c463-76b7-478a-a42f-dfaeccf378a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:33.524404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422c463-76b7-478a-a42f-dfaeccf378a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:33.525671 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422c463-76b7-478a-a42f-dfaeccf378a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:33.526174 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422c463-76b7-478a-a42f-dfaeccf378a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:33.527057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422c463-76b7-478a-a42f-dfaeccf378a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:33.529820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422c463-76b7-478a-a42f-dfaeccf378a7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:33.623923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1298/2603] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:31:33 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:31:33.624560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:33.643711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527332d0-219e-46e0-b8c3-68e615e8c04c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:33.714230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91beb2ff-dc5f-41d2-ad69-0ce4a61669e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:33.715196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beb2ff-dc5f-41d2-ad69-0ce4a61669e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:33.715675 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beb2ff-dc5f-41d2-ad69-0ce4a61669e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:33.715894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beb2ff-dc5f-41d2-ad69-0ce4a61669e2 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:33.717357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91beb2ff-dc5f-41d2-ad69-0ce4a61669e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:33.718582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91beb2ff-dc5f-41d2-ad69-0ce4a61669e2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:33.845757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1306/2604] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:31:33 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:31:33.851816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422c463-76b7-478a-a42f-dfaeccf378a7 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:33.864083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422c463-76b7-478a-a42f-dfaeccf378a7 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:33.869133 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24751de-9ad0-461e-a2de-52f321a0777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:33.869888 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24751de-9ad0-461e-a2de-52f321a0777b admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:33.870253 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24751de-9ad0-461e-a2de-52f321a0777b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:33.870848 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24751de-9ad0-461e-a2de-52f321a0777b admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:33.871882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24751de-9ad0-461e-a2de-52f321a0777b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:33.873585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24751de-9ad0-461e-a2de-52f321a0777b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:33.885320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1299/2605] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:31:33.890180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91beb2ff-dc5f-41d2-ad69-0ce4a61669e2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:33.903245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91beb2ff-dc5f-41d2-ad69-0ce4a61669e2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:33.908128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45520e-429c-4cfd-88e3-32056b17f7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:33.908749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45520e-429c-4cfd-88e3-32056b17f7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:33.909556 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45520e-429c-4cfd-88e3-32056b17f7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:33.909837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45520e-429c-4cfd-88e3-32056b17f7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:33.910535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45520e-429c-4cfd-88e3-32056b17f7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:33.912146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45520e-429c-4cfd-88e3-32056b17f7d3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:33.918216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:33.928694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1307/2606] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:31:33.936707 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24751de-9ad0-461e-a2de-52f321a0777b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:33.949956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24751de-9ad0-461e-a2de-52f321a0777b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:33.958024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10043c4-b627-4719-9e57-b8fbcfac2cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:33.958744 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10043c4-b627-4719-9e57-b8fbcfac2cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:33.959031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10043c4-b627-4719-9e57-b8fbcfac2cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:33.959308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10043c4-b627-4719-9e57-b8fbcfac2cb9 admin admin] PATH_INFO: `/v3/projects/fab8c79277ea4dd9b8cfd21921cefdec/users/77c5b8b1948d480ca86a600e93a3c16f/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:33.977986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10043c4-b627-4719-9e57-b8fbcfac2cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab8c79277ea4dd9b8cfd21921cefdec, user_id=77c5b8b1948d480ca86a600e93a3c16f, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:33.980701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10043c4-b627-4719-9e57-b8fbcfac2cb9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:34.094840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1300/2607] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:31:33 2022] PUT /identity/v3/projects/fab8c79277ea4dd9b8cfd21921cefdec/users/77c5b8b1948d480ca86a600e93a3c16f/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:34.097059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813093-05b3-43ca-bd6f-cf7efcce12c7 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:34.097376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813093-05b3-43ca-bd6f-cf7efcce12c7 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:34.097650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813093-05b3-43ca-bd6f-cf7efcce12c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:34.133666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-7b813093-05b3-43ca-bd6f-cf7efcce12c7 None None] MFA Rules not processed for user `77c5b8b1948d480ca86a600e93a3c16f`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:34.134952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7b813093-05b3-43ca-bd6f-cf7efcce12c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:34.207685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1308/2608] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:34.212382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10043c4-b627-4719-9e57-b8fbcfac2cb9 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:34.287678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10043c4-b627-4719-9e57-b8fbcfac2cb9 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:34.296911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820987-fafc-4c43-9511-ea08bc3a61b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:34.297830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820987-fafc-4c43-9511-ea08bc3a61b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:34.298127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820987-fafc-4c43-9511-ea08bc3a61b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:34.298455 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820987-fafc-4c43-9511-ea08bc3a61b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:34.299109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18820987-fafc-4c43-9511-ea08bc3a61b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:34.301056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18820987-fafc-4c43-9511-ea08bc3a61b8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:34.390727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1301/2609] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:31:34 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:31:34.395101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b813093-05b3-43ca-bd6f-cf7efcce12c7 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:34.408612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b813093-05b3-43ca-bd6f-cf7efcce12c7 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:34.473939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa41dbf-6d01-4385-8457-569bf3ddf02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:34.474643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa41dbf-6d01-4385-8457-569bf3ddf02d admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:34.474926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa41dbf-6d01-4385-8457-569bf3ddf02d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:34.475197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa41dbf-6d01-4385-8457-569bf3ddf02d admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:34.475884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa41dbf-6d01-4385-8457-569bf3ddf02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:34.477912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa41dbf-6d01-4385-8457-569bf3ddf02d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:34.582648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1309/2610] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:31:34 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:31:34.584861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820987-fafc-4c43-9511-ea08bc3a61b8 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:34.605004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820987-fafc-4c43-9511-ea08bc3a61b8 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:34.609880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989f7f1-938d-4d0f-b9d3-fd22c7edd112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:34.610530 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989f7f1-938d-4d0f-b9d3-fd22c7edd112 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:34.610816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989f7f1-938d-4d0f-b9d3-fd22c7edd112 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:34.611094 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989f7f1-938d-4d0f-b9d3-fd22c7edd112 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:34.611761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989f7f1-938d-4d0f-b9d3-fd22c7edd112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:34.613876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989f7f1-938d-4d0f-b9d3-fd22c7edd112 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:34.626171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1302/2611] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:31:34.631252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa41dbf-6d01-4385-8457-569bf3ddf02d admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:34.644844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa41dbf-6d01-4385-8457-569bf3ddf02d admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:34.651504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15422b43-bcd7-49b0-a45b-6776ea9c2c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:34.652492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15422b43-bcd7-49b0-a45b-6776ea9c2c38 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:34.652936 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15422b43-bcd7-49b0-a45b-6776ea9c2c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:34.653375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15422b43-bcd7-49b0-a45b-6776ea9c2c38 admin admin] PATH_INFO: `/v3/projects/d8836eab8f6d4d3cb1085c5c8c15b437/users/dcbc0dee558b441f8e5b5efc78748de8/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:34.676021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15422b43-bcd7-49b0-a45b-6776ea9c2c38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8836eab8f6d4d3cb1085c5c8c15b437, user_id=dcbc0dee558b441f8e5b5efc78748de8, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:34.678856 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15422b43-bcd7-49b0-a45b-6776ea9c2c38 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:34.760821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1310/2612] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:31:34 2022] PUT /identity/v3/projects/d8836eab8f6d4d3cb1085c5c8c15b437/users/dcbc0dee558b441f8e5b5efc78748de8/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:34.765647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989f7f1-938d-4d0f-b9d3-fd22c7edd112 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:34.777242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989f7f1-938d-4d0f-b9d3-fd22c7edd112 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:34.849334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f4ddb-7a61-4565-aabc-f059795092ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:34.849951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f4ddb-7a61-4565-aabc-f059795092ed admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:34.850269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f4ddb-7a61-4565-aabc-f059795092ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:34.850554 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f4ddb-7a61-4565-aabc-f059795092ed admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:34.851197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f4ddb-7a61-4565-aabc-f059795092ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:34.852796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f4ddb-7a61-4565-aabc-f059795092ed admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:34.870935 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1303/2613] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:31:34.872231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15422b43-bcd7-49b0-a45b-6776ea9c2c38 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:34.883416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15422b43-bcd7-49b0-a45b-6776ea9c2c38 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:34.894092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:34.894990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:34.895371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:34.895708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] PATH_INFO: `/v3/projects/d8836eab8f6d4d3cb1085c5c8c15b437/users/dcbc0dee558b441f8e5b5efc78748de8/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:34.897810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8836eab8f6d4d3cb1085c5c8c15b437, user_id=dcbc0dee558b441f8e5b5efc78748de8, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:34.900641 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:34.926913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1311/2614] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:31:34 2022] PUT /identity/v3/projects/d8836eab8f6d4d3cb1085c5c8c15b437/users/dcbc0dee558b441f8e5b5efc78748de8/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:34.927797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bac1c-9ec7-40ce-99b2-c4e632fde263 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:34.928088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bac1c-9ec7-40ce-99b2-c4e632fde263 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:34.928365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bac1c-9ec7-40ce-99b2-c4e632fde263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:34.936933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:34.965050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-165bac1c-9ec7-40ce-99b2-c4e632fde263 None None] MFA Rules not processed for user `dcbc0dee558b441f8e5b5efc78748de8`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:34.966975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-165bac1c-9ec7-40ce-99b2-c4e632fde263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:35.048139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1304/2615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:31:35.062354 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:35.064338 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:35.122246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:35.138733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa8b1-30f7-4a79-be7d-05e0406c10a8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:35.145141 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97664476-3194-42bc-9ebd-05baeadca2c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:35.145916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664476-3194-42bc-9ebd-05baeadca2c1 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:35.146307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664476-3194-42bc-9ebd-05baeadca2c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:35.146654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664476-3194-42bc-9ebd-05baeadca2c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:35.157394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97664476-3194-42bc-9ebd-05baeadca2c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:35.159494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97664476-3194-42bc-9ebd-05baeadca2c1 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:35.170224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1312/2616] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:35.244288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bac1c-9ec7-40ce-99b2-c4e632fde263 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:35.318117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bac1c-9ec7-40ce-99b2-c4e632fde263 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:35.322956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:35.323599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:35.323880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:35.324143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] PATH_INFO: `/v3/users/77c5b8b1948d480ca86a600e93a3c16f` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:35.325278 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c5b8b1948d480ca86a600e93a3c16f)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:35.327225 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:35.889061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c5b8b1948d480ca86a600e93a3c16f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:35.965527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c5b8b1948d480ca86a600e93a3c16f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:35.966209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c5b8b1948d480ca86a600e93a3c16f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:35.966912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c5b8b1948d480ca86a600e93a3c16f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:35.976666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c5b8b1948d480ca86a600e93a3c16f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:35.998989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1305/2617] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:31:35 2022] DELETE /identity/v3/users/77c5b8b1948d480ca86a600e93a3c16f => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:36.004425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc573-5efd-4c9f-9e4a-3aaa0fe74135 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:36.005336 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc573-5efd-4c9f-9e4a-3aaa0fe74135 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:36.005777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc573-5efd-4c9f-9e4a-3aaa0fe74135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:36.027689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-dfbfc573-5efd-4c9f-9e4a-3aaa0fe74135 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:36.028979 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-dfbfc573-5efd-4c9f-9e4a-3aaa0fe74135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:36.099882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1313/2618] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:36.110405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:36.112920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:36.126715 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:36.188437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe7126-0477-4ce0-86fc-90cc236010a1 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:36.193348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658091a6-6538-43a8-aaf5-fc856a9068fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:36.193958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658091a6-6538-43a8-aaf5-fc856a9068fd service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:36.194324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658091a6-6538-43a8-aaf5-fc856a9068fd service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:36.194581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658091a6-6538-43a8-aaf5-fc856a9068fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:36.204551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658091a6-6538-43a8-aaf5-fc856a9068fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:36.206029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658091a6-6538-43a8-aaf5-fc856a9068fd service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:36.217572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1306/2619] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:36.716202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfc573-5efd-4c9f-9e4a-3aaa0fe74135 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:36.718347 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-dfbfc573-5efd-4c9f-9e4a-3aaa0fe74135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:36.741011 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfc573-5efd-4c9f-9e4a-3aaa0fe74135 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:36.749343 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:36.750166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:36.750540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:36.750907 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] PATH_INFO: `/v3/projects/fab8c79277ea4dd9b8cfd21921cefdec` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:36.753798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab8c79277ea4dd9b8cfd21921cefdec)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:36.756351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:36.947203 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab8c79277ea4dd9b8cfd21921cefdec', 'request_id': 'req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:36.967214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab8c79277ea4dd9b8cfd21921cefdec', 'request_id': 'req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.045339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab8c79277ea4dd9b8cfd21921cefdec', 'request_id': 'req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.065873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] The token cache is being invalidate because project fab8c79277ea4dd9b8cfd21921cefdec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:31:37.066243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.071811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.075476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1314/2620] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:31:36 2022] DELETE /identity/v3/projects/fab8c79277ea4dd9b8cfd21921cefdec => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:37.080238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658091a6-6538-43a8-aaf5-fc856a9068fd service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:37.100416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658091a6-6538-43a8-aaf5-fc856a9068fd service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:37.199377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:37.200018 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:37.200302 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:37.200575 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] PATH_INFO: `/v3/users/dcbc0dee558b441f8e5b5efc78748de8` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:37.201712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcbc0dee558b441f8e5b5efc78748de8)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:37.203705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:37.428113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcbc0dee558b441f8e5b5efc78748de8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.507924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcbc0dee558b441f8e5b5efc78748de8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.508596 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcbc0dee558b441f8e5b5efc78748de8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.509207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcbc0dee558b441f8e5b5efc78748de8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.517703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9b0a4e89-0a32-4a0a-b104-11e57ca0f10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcbc0dee558b441f8e5b5efc78748de8'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:37.540799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1307/2621] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:31:37 2022] DELETE /identity/v3/users/dcbc0dee558b441f8e5b5efc78748de8 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:38.025576 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:38.027441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:38.062378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ec0c3-9975-40dc-ae0e-aa9b50918479 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:38.141863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:38.143536 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:38.143908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:38.144179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] PATH_INFO: `/v3/projects/d8836eab8f6d4d3cb1085c5c8c15b437` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:38.145335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8836eab8f6d4d3cb1085c5c8c15b437)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:38.147847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:38.160142 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:38.332968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8836eab8f6d4d3cb1085c5c8c15b437', 'request_id': 'req-1e4373ba-90de-497f-a9b0-8bf3b2c19254'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:38.351156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8836eab8f6d4d3cb1085c5c8c15b437', 'request_id': 'req-1e4373ba-90de-497f-a9b0-8bf3b2c19254'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:38.411332 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8836eab8f6d4d3cb1085c5c8c15b437', 'request_id': 'req-1e4373ba-90de-497f-a9b0-8bf3b2c19254'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:38.428355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] The token cache is being invalidate because project d8836eab8f6d4d3cb1085c5c8c15b437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:31:38.428736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:38.429597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1e4373ba-90de-497f-a9b0-8bf3b2c19254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:38.431674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1315/2622] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:31:38 2022] DELETE /identity/v3/projects/d8836eab8f6d4d3cb1085c5c8c15b437 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:38.438727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ac633-0e1b-4ba4-8aa5-207064a91277 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:38.440389 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ac633-0e1b-4ba4-8aa5-207064a91277 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:38.442284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ac633-0e1b-4ba4-8aa5-207064a91277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:38.474125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-324ac633-0e1b-4ba4-8aa5-207064a91277 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:38.476279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-324ac633-0e1b-4ba4-8aa5-207064a91277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:38.536150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:38.579279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1308/2623] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:38.587400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229e9c5-39dd-4202-a575-4c483358d59b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:38.587788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229e9c5-39dd-4202-a575-4c483358d59b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:38.588138 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229e9c5-39dd-4202-a575-4c483358d59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:38.617604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-6229e9c5-39dd-4202-a575-4c483358d59b None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:38.618776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6229e9c5-39dd-4202-a575-4c483358d59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:38.627040 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1316/2624] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:38.631623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ac633-0e1b-4ba4-8aa5-207064a91277 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:38.649621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ac633-0e1b-4ba4-8aa5-207064a91277 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:38.654743 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:38.655522 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:38.655813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:38.656091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:38.656815 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:31:38.657124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:38.659523 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:38.677748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1309/2625] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:31:38.679061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229e9c5-39dd-4202-a575-4c483358d59b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:38.690126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229e9c5-39dd-4202-a575-4c483358d59b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:38.696428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cd582-12e9-4303-a4a9-ddb2bb1461a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:38.697059 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cd582-12e9-4303-a4a9-ddb2bb1461a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:38.697339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cd582-12e9-4303-a4a9-ddb2bb1461a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:38.697612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cd582-12e9-4303-a4a9-ddb2bb1461a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:38.698291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cd582-12e9-4303-a4a9-ddb2bb1461a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:38.700231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cd582-12e9-4303-a4a9-ddb2bb1461a8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:39.029727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1317/2626] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:31:38 2022] POST /identity/v3/projects => generated 407 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:31:39.034486 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:39.044930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74ce7-6258-4280-8d59-b48d5eaf90e4 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:39.105220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362628cb-9e6f-4295-8a60-1c9907078efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:39.105853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362628cb-9e6f-4295-8a60-1c9907078efc admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:39.106188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362628cb-9e6f-4295-8a60-1c9907078efc admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:39.115919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362628cb-9e6f-4295-8a60-1c9907078efc admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:39.115919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362628cb-9e6f-4295-8a60-1c9907078efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:39.115919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362628cb-9e6f-4295-8a60-1c9907078efc admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:39.268608 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1310/2627] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:31:39 2022] POST /identity/v3/users => generated 304 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:31:39.272773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cd582-12e9-4303-a4a9-ddb2bb1461a8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:39.283294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cd582-12e9-4303-a4a9-ddb2bb1461a8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:39.292171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c50b4-57d8-4960-90d7-e885a2eae2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:39.292804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c50b4-57d8-4960-90d7-e885a2eae2bf admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:39.293145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c50b4-57d8-4960-90d7-e885a2eae2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:39.293426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c50b4-57d8-4960-90d7-e885a2eae2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:39.294083 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c50b4-57d8-4960-90d7-e885a2eae2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:39.295761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c50b4-57d8-4960-90d7-e885a2eae2bf admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:39.307998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1318/2628] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:31:39.312226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362628cb-9e6f-4295-8a60-1c9907078efc admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:39.322351 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362628cb-9e6f-4295-8a60-1c9907078efc admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:39.326892 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6397c-dd36-49a5-82f8-615c1478ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:39.327552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6397c-dd36-49a5-82f8-615c1478ec26 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:39.327833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6397c-dd36-49a5-82f8-615c1478ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:39.328106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6397c-dd36-49a5-82f8-615c1478ec26 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:39.328745 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6397c-dd36-49a5-82f8-615c1478ec26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:39.330405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6397c-dd36-49a5-82f8-615c1478ec26 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:39.345520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1311/2629] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:31:39.349849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c50b4-57d8-4960-90d7-e885a2eae2bf admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:39.360711 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c50b4-57d8-4960-90d7-e885a2eae2bf admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:39.366384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2e0-afd2-4c8c-bc34-45e7a7215934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:39.367535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2e0-afd2-4c8c-bc34-45e7a7215934 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:39.367535 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2e0-afd2-4c8c-bc34-45e7a7215934 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:39.367793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2e0-afd2-4c8c-bc34-45e7a7215934 admin admin] PATH_INFO: `/v3/projects/1c51f52b87fc4ff0a1f219f9521fab58/users/732357c85dd642349d312dc5d4551c3f/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:39.385154 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86b2e0-afd2-4c8c-bc34-45e7a7215934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c51f52b87fc4ff0a1f219f9521fab58, user_id=732357c85dd642349d312dc5d4551c3f, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:39.387937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86b2e0-afd2-4c8c-bc34-45e7a7215934 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:39.474511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1319/2630] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:31:39 2022] PUT /identity/v3/projects/1c51f52b87fc4ff0a1f219f9521fab58/users/732357c85dd642349d312dc5d4551c3f/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:39.483924 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba1ee4-b7af-4de2-bb4e-b04075fe93ee None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:39.485807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba1ee4-b7af-4de2-bb4e-b04075fe93ee None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:39.486441 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba1ee4-b7af-4de2-bb4e-b04075fe93ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:39.525882 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-06ba1ee4-b7af-4de2-bb4e-b04075fe93ee None None] MFA Rules not processed for user `732357c85dd642349d312dc5d4551c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:39.527217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-06ba1ee4-b7af-4de2-bb4e-b04075fe93ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:39.673490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1312/2631] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:39 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:39.675386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2e0-afd2-4c8c-bc34-45e7a7215934 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:39.748457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2e0-afd2-4c8c-bc34-45e7a7215934 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:39.754015 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6850f0-cbd2-48c7-b9be-3029e57b3da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:39.754665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6850f0-cbd2-48c7-b9be-3029e57b3da9 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:39.754948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6850f0-cbd2-48c7-b9be-3029e57b3da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:39.755220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6850f0-cbd2-48c7-b9be-3029e57b3da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:39.755859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6850f0-cbd2-48c7-b9be-3029e57b3da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:39.757839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6850f0-cbd2-48c7-b9be-3029e57b3da9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:39.859042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1320/2632] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:31:39 2022] POST /identity/v3/projects => generated 409 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:31:39.863350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba1ee4-b7af-4de2-bb4e-b04075fe93ee None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:39.874631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba1ee4-b7af-4de2-bb4e-b04075fe93ee None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:39.899749 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:39.933961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd2208-5d69-4d77-b518-789dc1dd3b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:39.934621 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd2208-5d69-4d77-b518-789dc1dd3b58 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:39.934907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd2208-5d69-4d77-b518-789dc1dd3b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:39.935200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd2208-5d69-4d77-b518-789dc1dd3b58 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:39.935843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd2208-5d69-4d77-b518-789dc1dd3b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:39.937770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd2208-5d69-4d77-b518-789dc1dd3b58 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:40.051050 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1313/2633] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:31:39 2022] POST /identity/v3/users => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:31:40.055511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6850f0-cbd2-48c7-b9be-3029e57b3da9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:40.067497 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6850f0-cbd2-48c7-b9be-3029e57b3da9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:40.072523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5d280-52b1-4931-9f7e-3214630477d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:40.073191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d280-52b1-4931-9f7e-3214630477d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:40.077276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d280-52b1-4931-9f7e-3214630477d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:40.077276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d280-52b1-4931-9f7e-3214630477d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:40.077276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5d280-52b1-4931-9f7e-3214630477d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:40.077276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5d280-52b1-4931-9f7e-3214630477d4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:40.089520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1321/2634] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:31:40.093740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd2208-5d69-4d77-b518-789dc1dd3b58 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:40.104508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd2208-5d69-4d77-b518-789dc1dd3b58 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:40.109152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f895965-9c6a-4a90-8e45-21ca029387eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:40.109811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f895965-9c6a-4a90-8e45-21ca029387eb admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:40.110096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f895965-9c6a-4a90-8e45-21ca029387eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:40.110414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f895965-9c6a-4a90-8e45-21ca029387eb admin admin] PATH_INFO: `/v3/projects/fbd31041024d44299ddc17aeb7882087/users/d72d04ab8f49408dab354675ded653a0/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:40.125581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f895965-9c6a-4a90-8e45-21ca029387eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd31041024d44299ddc17aeb7882087, user_id=d72d04ab8f49408dab354675ded653a0, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:40.128840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f895965-9c6a-4a90-8e45-21ca029387eb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:40.416784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1314/2635] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:31:40 2022] PUT /identity/v3/projects/fbd31041024d44299ddc17aeb7882087/users/d72d04ab8f49408dab354675ded653a0/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:40.420951 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5d280-52b1-4931-9f7e-3214630477d4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:40.431968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5d280-52b1-4931-9f7e-3214630477d4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:40.436909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:40.494660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cb4f9-5947-4eff-b54e-7fbff732f3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:40.495271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb4f9-5947-4eff-b54e-7fbff732f3cb admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:40.495549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb4f9-5947-4eff-b54e-7fbff732f3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:40.495877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb4f9-5947-4eff-b54e-7fbff732f3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:40.496549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cb4f9-5947-4eff-b54e-7fbff732f3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:40.498259 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cb4f9-5947-4eff-b54e-7fbff732f3cb admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:40.510915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1322/2636] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:31:40.513288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f895965-9c6a-4a90-8e45-21ca029387eb admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:40.523910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f895965-9c6a-4a90-8e45-21ca029387eb admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:40.528947 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:40.529611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:40.529885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:40.530199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] PATH_INFO: `/v3/projects/fbd31041024d44299ddc17aeb7882087/users/d72d04ab8f49408dab354675ded653a0/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:40.532053 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd31041024d44299ddc17aeb7882087, user_id=d72d04ab8f49408dab354675ded653a0, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:40.534779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:40.545158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1315/2637] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:31:40 2022] PUT /identity/v3/projects/fbd31041024d44299ddc17aeb7882087/users/d72d04ab8f49408dab354675ded653a0/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:40.557881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8980788-7205-4231-8bfc-659bc0a1637c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:40.558198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8980788-7205-4231-8bfc-659bc0a1637c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:40.558490 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8980788-7205-4231-8bfc-659bc0a1637c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:40.587231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c8980788-7205-4231-8bfc-659bc0a1637c None None] MFA Rules not processed for user `d72d04ab8f49408dab354675ded653a0`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:40.588446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c8980788-7205-4231-8bfc-659bc0a1637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:40.670224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1323/2638] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:40 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:40.686355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:40.686355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:40.761768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6da4d-1e89-4af1-a93f-8c0566fa8870 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:40.766693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1585619-365d-40df-88a8-d75172222c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:40.767315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1585619-365d-40df-88a8-d75172222c05 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:40.767597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1585619-365d-40df-88a8-d75172222c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:40.767871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1585619-365d-40df-88a8-d75172222c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:40.778880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1585619-365d-40df-88a8-d75172222c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:40.780761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1585619-365d-40df-88a8-d75172222c05 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:40.793075 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1316/2639] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:40 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:41.220778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8980788-7205-4231-8bfc-659bc0a1637c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:41.231088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8980788-7205-4231-8bfc-659bc0a1637c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:41.234796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76fd34e-e634-4f26-a769-061770e6a0d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:41.235394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fd34e-e634-4f26-a769-061770e6a0d1 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:41.235674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fd34e-e634-4f26-a769-061770e6a0d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:41.235946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fd34e-e634-4f26-a769-061770e6a0d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:41.294423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76fd34e-e634-4f26-a769-061770e6a0d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:41.296033 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76fd34e-e634-4f26-a769-061770e6a0d1 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:41.304983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1324/2640] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:41 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:50.105965 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934cf79-21ad-4575-9631-1d74082a8810 None None] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:50.106684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934cf79-21ad-4575-9631-1d74082a8810 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:50.106684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934cf79-21ad-4575-9631-1d74082a8810 None None] PATH_INFO: `/` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:50.108373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1317/2641] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 14:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:31:50.113930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dafbf-8b27-422e-9b4e-2915c406233e None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:50.114449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dafbf-8b27-422e-9b4e-2915c406233e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:50.114582 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dafbf-8b27-422e-9b4e-2915c406233e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:50.160486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:50.178982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-358dafbf-8b27-422e-9b4e-2915c406233e None None] MFA Rules not processed for user `c24d70e571c64f018aa199c4c08d153e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:50.180219 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-358dafbf-8b27-422e-9b4e-2915c406233e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:50.254356 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1325/2642] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:50.267076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934cf79-21ad-4575-9631-1d74082a8810 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:50.269451 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a934cf79-21ad-4575-9631-1d74082a8810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:50.350296 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934cf79-21ad-4575-9631-1d74082a8810 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:50.353808 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545c125-ba90-473b-934b-a6a139cd8848 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c07c85b63514d10bb17b836c60f2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:50.354452 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545c125-ba90-473b-934b-a6a139cd8848 service glance] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:50.354726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545c125-ba90-473b-934b-a6a139cd8848 service glance] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:50.354998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545c125-ba90-473b-934b-a6a139cd8848 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:50.366748 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545c125-ba90-473b-934b-a6a139cd8848 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:50.368189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545c125-ba90-473b-934b-a6a139cd8848 service glance] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:50.379312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1318/2643] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:50.577725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde32ec1-60a0-4e33-9b46-038ff203fd2c None None] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:50.578008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde32ec1-60a0-4e33-9b46-038ff203fd2c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:50.578316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde32ec1-60a0-4e33-9b46-038ff203fd2c None None] PATH_INFO: `/` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:50.579662 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1326/2644] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 14:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:31:50.585078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df834d5b-1658-4f13-a507-6b9fb019a379 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:50.585377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df834d5b-1658-4f13-a507-6b9fb019a379 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:50.585681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df834d5b-1658-4f13-a507-6b9fb019a379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:50.594920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:50.625016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-df834d5b-1658-4f13-a507-6b9fb019a379 None None] MFA Rules not processed for user `a71d353c1bd849028930261e9306fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:50.626281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-df834d5b-1658-4f13-a507-6b9fb019a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:50.680190 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1319/2645] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:31:50.693060 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde32ec1-60a0-4e33-9b46-038ff203fd2c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:50.695155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bde32ec1-60a0-4e33-9b46-038ff203fd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:50.709286 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde32ec1-60a0-4e33-9b46-038ff203fd2c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:50.712823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bbb6d-d7ea-4e62-9900-0c40e13170e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:50.713425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bbb6d-d7ea-4e62-9900-0c40e13170e9 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:50.713702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bbb6d-d7ea-4e62-9900-0c40e13170e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:50.713967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bbb6d-d7ea-4e62-9900-0c40e13170e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:50.727344 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bbb6d-d7ea-4e62-9900-0c40e13170e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:50.729353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bbb6d-d7ea-4e62-9900-0c40e13170e9 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:50.739996 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1327/2646] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:50.826267 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df834d5b-1658-4f13-a507-6b9fb019a379 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:50.828143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-df834d5b-1658-4f13-a507-6b9fb019a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:50.838603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df834d5b-1658-4f13-a507-6b9fb019a379 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:50.842240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939621b-858e-4736-8c30-92d5e210d1b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:50.842847 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939621b-858e-4736-8c30-92d5e210d1b2 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:50.843153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939621b-858e-4736-8c30-92d5e210d1b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:50.843469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939621b-858e-4736-8c30-92d5e210d1b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:50.845158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c939621b-858e-4736-8c30-92d5e210d1b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:50.859629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939621b-858e-4736-8c30-92d5e210d1b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:50.861060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939621b-858e-4736-8c30-92d5e210d1b2 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:50.872449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1320/2647] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:51.003908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bbb6d-d7ea-4e62-9900-0c40e13170e9 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:51.078070 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bbb6d-d7ea-4e62-9900-0c40e13170e9 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:51.083553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:51.084196 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:51.084485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:51.084758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] PATH_INFO: `/v3/users/732357c85dd642349d312dc5d4551c3f` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:51.085900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] RBAC: Authorizing `identity:delete_user(user_id=732357c85dd642349d312dc5d4551c3f)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:51.087937 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:51.289289 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '732357c85dd642349d312dc5d4551c3f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:51.348180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '732357c85dd642349d312dc5d4551c3f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:51.349502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '732357c85dd642349d312dc5d4551c3f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:51.383055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '732357c85dd642349d312dc5d4551c3f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:51.391606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '732357c85dd642349d312dc5d4551c3f'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:51.397863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1328/2648] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:31:51 2022] DELETE /identity/v3/users/732357c85dd642349d312dc5d4551c3f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:51.402658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94a072-8d2c-40cf-8063-ad03af0a0b01 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:51.403193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94a072-8d2c-40cf-8063-ad03af0a0b01 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:51.403405 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94a072-8d2c-40cf-8063-ad03af0a0b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:51.423867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-6a94a072-8d2c-40cf-8063-ad03af0a0b01 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:51.425582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6a94a072-8d2c-40cf-8063-ad03af0a0b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:51.484563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1321/2649] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:51.495032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:51.496462 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:51.501879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:51.564359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a30104-9a45-4dca-acb2-189e3c07495a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:51.575743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd09055-ff0c-4613-b356-a59212e8f9f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:51.576393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd09055-ff0c-4613-b356-a59212e8f9f9 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:51.576676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd09055-ff0c-4613-b356-a59212e8f9f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:51.576949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd09055-ff0c-4613-b356-a59212e8f9f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:51.586806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd09055-ff0c-4613-b356-a59212e8f9f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:51.588243 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd09055-ff0c-4613-b356-a59212e8f9f9 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:51.600182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1329/2650] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:31:52.059559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94a072-8d2c-40cf-8063-ad03af0a0b01 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:52.061632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6a94a072-8d2c-40cf-8063-ad03af0a0b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:52.076469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94a072-8d2c-40cf-8063-ad03af0a0b01 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:52.081681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:52.082348 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:52.082632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:52.082907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] PATH_INFO: `/v3/projects/1c51f52b87fc4ff0a1f219f9521fab58` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:52.084033 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c51f52b87fc4ff0a1f219f9521fab58)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:52.085930 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:52.216228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c51f52b87fc4ff0a1f219f9521fab58', 'request_id': 'req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.223500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c51f52b87fc4ff0a1f219f9521fab58', 'request_id': 'req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.296710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c51f52b87fc4ff0a1f219f9521fab58', 'request_id': 'req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.307802 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] The token cache is being invalidate because project 1c51f52b87fc4ff0a1f219f9521fab58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:31:52.308036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.308674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.310187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1322/2651] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:31:52 2022] DELETE /identity/v3/projects/1c51f52b87fc4ff0a1f219f9521fab58 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:52.314348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd09055-ff0c-4613-b356-a59212e8f9f9 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:52.323986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd09055-ff0c-4613-b356-a59212e8f9f9 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:52.391088 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:52.391742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:52.392055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:52.392331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] PATH_INFO: `/v3/users/d72d04ab8f49408dab354675ded653a0` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:52.393518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=d72d04ab8f49408dab354675ded653a0)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:52.395475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:52.548619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd72d04ab8f49408dab354675ded653a0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.616733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd72d04ab8f49408dab354675ded653a0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.617394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd72d04ab8f49408dab354675ded653a0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.635212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd72d04ab8f49408dab354675ded653a0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.645065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-ed82314a-004c-4540-82b0-cd1d53b879ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd72d04ab8f49408dab354675ded653a0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:52.651198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1330/2652] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:31:52 2022] DELETE /identity/v3/users/d72d04ab8f49408dab354675ded653a0 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:53.227550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:53.229299 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:53.249885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269282cc-0d68-4bf2-9d2b-4b8f5e49b1d1 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:53.258693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:53.321256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:53.321916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:53.322227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:53.322506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] PATH_INFO: `/v3/projects/fbd31041024d44299ddc17aeb7882087` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:53.323729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd31041024d44299ddc17aeb7882087)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:53.326546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:53.442625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd31041024d44299ddc17aeb7882087', 'request_id': 'req-02d61cad-6f3c-4688-84a8-aebfe46f244b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:53.448869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd31041024d44299ddc17aeb7882087', 'request_id': 'req-02d61cad-6f3c-4688-84a8-aebfe46f244b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:53.525111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd31041024d44299ddc17aeb7882087', 'request_id': 'req-02d61cad-6f3c-4688-84a8-aebfe46f244b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:53.537955 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] The token cache is being invalidate because project fbd31041024d44299ddc17aeb7882087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:31:53.538625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:53.539084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-02d61cad-6f3c-4688-84a8-aebfe46f244b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:31:53.540646 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1323/2653] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:31:53 2022] DELETE /identity/v3/projects/fbd31041024d44299ddc17aeb7882087 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:31:53.546333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1468b-c1fd-4f6d-a837-8b637124ef70 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:53.546833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1468b-c1fd-4f6d-a837-8b637124ef70 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:53.546953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1468b-c1fd-4f6d-a837-8b637124ef70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:53.567224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e1b1468b-c1fd-4f6d-a837-8b637124ef70 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:53.568430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e1b1468b-c1fd-4f6d-a837-8b637124ef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:53.587704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:53.636670 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1331/2654] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:53.644939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b8d7e-ad63-4d4d-8f9d-d3c77fb1513e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:53.645256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b8d7e-ad63-4d4d-8f9d-d3c77fb1513e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:53.645537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b8d7e-ad63-4d4d-8f9d-d3c77fb1513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:53.669380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-733b8d7e-ad63-4d4d-8f9d-d3c77fb1513e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:53.671500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-733b8d7e-ad63-4d4d-8f9d-d3c77fb1513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:53.682715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbc5a8-06eb-4c1b-a2be-2b7221eb03af None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:53.682947 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbc5a8-06eb-4c1b-a2be-2b7221eb03af None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:53.683234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbc5a8-06eb-4c1b-a2be-2b7221eb03af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:53.685742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1324/2655] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:53.690306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b8d7e-ad63-4d4d-8f9d-d3c77fb1513e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:53.703845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b8d7e-ad63-4d4d-8f9d-d3c77fb1513e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:53.705737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-eccbc5a8-06eb-4c1b-a2be-2b7221eb03af None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:53.707199 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-eccbc5a8-06eb-4c1b-a2be-2b7221eb03af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:53.711226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:53.711226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:53.711226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:53.711226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:53.711662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:31:53.711662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:53.713360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:53.734263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1325/2656] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:31:53.739444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:53.750944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce92f11-1275-40b4-975c-452578f758db admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:53.760584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9374b-6e60-4db8-aaf7-eca9337378c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:53.760584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9374b-6e60-4db8-aaf7-eca9337378c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:53.760584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9374b-6e60-4db8-aaf7-eca9337378c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:53.761042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9374b-6e60-4db8-aaf7-eca9337378c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:53.761464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9374b-6e60-4db8-aaf7-eca9337378c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:53.763458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9374b-6e60-4db8-aaf7-eca9337378c4 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:53.779352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1332/2657] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 14:31:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:31:53.791295 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbc5a8-06eb-4c1b-a2be-2b7221eb03af None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:53.792798 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-eccbc5a8-06eb-4c1b-a2be-2b7221eb03af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:53.864523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbc5a8-06eb-4c1b-a2be-2b7221eb03af None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:53.864763 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1326/2658] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:31:53 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:31:53.868641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9374b-6e60-4db8-aaf7-eca9337378c4 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:53.928520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc050-3b39-476e-b94a-deb9d1b0e948 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:53.929053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc050-3b39-476e-b94a-deb9d1b0e948 service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:53.929333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc050-3b39-476e-b94a-deb9d1b0e948 service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:53.929606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc050-3b39-476e-b94a-deb9d1b0e948 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:53.932900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9374b-6e60-4db8-aaf7-eca9337378c4 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:53.938350 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfd75-b0af-4ba2-ab76-d279d571566d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:53.938983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfd75-b0af-4ba2-ab76-d279d571566d admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:53.939266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfd75-b0af-4ba2-ab76-d279d571566d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:53.940613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfd75-b0af-4ba2-ab76-d279d571566d admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:53.941293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddfd75-b0af-4ba2-ab76-d279d571566d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:53.943020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:53.943588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddfd75-b0af-4ba2-ab76-d279d571566d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:53.951832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:54.056903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1327/2659] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:31:53 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:31:54.061287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfd75-b0af-4ba2-ab76-d279d571566d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:54.071991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfd75-b0af-4ba2-ab76-d279d571566d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:54.079901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab54057-c019-40fd-8882-121f318105d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:54.081538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab54057-c019-40fd-8882-121f318105d1 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:54.081538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab54057-c019-40fd-8882-121f318105d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:54.081538 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab54057-c019-40fd-8882-121f318105d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:54.082268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab54057-c019-40fd-8882-121f318105d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:54.083999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab54057-c019-40fd-8882-121f318105d1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:54.095119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1328/2660] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:31:54.100987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab54057-c019-40fd-8882-121f318105d1 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:54.112255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab54057-c019-40fd-8882-121f318105d1 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:54.116965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc050-3b39-476e-b94a-deb9d1b0e948 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:54.116965 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc050-3b39-476e-b94a-deb9d1b0e948 service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:54.117385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8dce3-a536-4e4e-82e2-d0717dabf454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:54.117754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8dce3-a536-4e4e-82e2-d0717dabf454 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:54.118040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8dce3-a536-4e4e-82e2-d0717dabf454 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:54.118489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8dce3-a536-4e4e-82e2-d0717dabf454 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:54.119142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8dce3-a536-4e4e-82e2-d0717dabf454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:54.120733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8dce3-a536-4e4e-82e2-d0717dabf454 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:54.123970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1333/2661] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:31:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 14:31:54.133242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1329/2662] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:31:54.138523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc050-3b39-476e-b94a-deb9d1b0e948 service swift] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:54.150167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc050-3b39-476e-b94a-deb9d1b0e948 service swift] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:54.155288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:54.155968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:54.156246 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:54.156514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] PATH_INFO: `/v3/projects/f6ca8ee07fe640d88546a4dfc09a08d9/users/b8a914e9648d403a86a694d79a09f1da/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:54.171854 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ca8ee07fe640d88546a4dfc09a08d9, user_id=b8a914e9648d403a86a694d79a09f1da, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:54.175055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:54.286461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1334/2663] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:31:54 2022] PUT /identity/v3/projects/f6ca8ee07fe640d88546a4dfc09a08d9/users/b8a914e9648d403a86a694d79a09f1da/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:31:54.298301 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ab9ee-bfd2-43ee-ac7d-3f7b8d65753d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:54.298909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ab9ee-bfd2-43ee-ac7d-3f7b8d65753d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:54.298909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ab9ee-bfd2-43ee-ac7d-3f7b8d65753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:54.329416 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-866ab9ee-bfd2-43ee-ac7d-3f7b8d65753d None None] MFA Rules not processed for user `b8a914e9648d403a86a694d79a09f1da`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:31:54.330397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-866ab9ee-bfd2-43ee-ac7d-3f7b8d65753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:54.458985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1330/2664] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:31:54.466428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:31:54.468551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:31:54.621521 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7029836-7dd8-40c4-a59b-cb9a97911d8d admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:31:54.638760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893cff7-d594-4f2b-afb4-6f23a35a463d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:31:54.639398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893cff7-d594-4f2b-afb4-6f23a35a463d service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:31:54.639939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893cff7-d594-4f2b-afb4-6f23a35a463d service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:31:54.640213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893cff7-d594-4f2b-afb4-6f23a35a463d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:31:54.655371 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893cff7-d594-4f2b-afb4-6f23a35a463d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:31:54.657750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893cff7-d594-4f2b-afb4-6f23a35a463d service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:31:54.683696 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1335/2665] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:31:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:04.444206 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:INSERT=3,keystone:SELECT=44 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:04.665673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=57,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:08.237101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ab9ee-bfd2-43ee-ac7d-3f7b8d65753d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:08.313109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ab9ee-bfd2-43ee-ac7d-3f7b8d65753d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:08.321062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:08.321710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:08.321995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:08.322303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] PATH_INFO: `/v3/users/b8a914e9648d403a86a694d79a09f1da` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:08.323433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8a914e9648d403a86a694d79a09f1da)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:08.325393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:08.513231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8a914e9648d403a86a694d79a09f1da'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:08.582280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8a914e9648d403a86a694d79a09f1da'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:08.583380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8a914e9648d403a86a694d79a09f1da'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:08.584001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8a914e9648d403a86a694d79a09f1da'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:08.592650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8a914e9648d403a86a694d79a09f1da'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:08.612052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1331/2666] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:32:08 2022] DELETE /identity/v3/users/b8a914e9648d403a86a694d79a09f1da => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:08.616603 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa699b-9e31-482a-9e4f-46840995af01 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:08.617121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa699b-9e31-482a-9e4f-46840995af01 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:08.617244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa699b-9e31-482a-9e4f-46840995af01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:08.636566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-77aa699b-9e31-482a-9e4f-46840995af01 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:08.640630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-77aa699b-9e31-482a-9e4f-46840995af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:08.707726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1336/2667] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:08.720242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:08.722275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:08.784128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84e9f-d284-48c2-9ab8-0aaa62878fcf admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:08.788755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd978a-a050-440d-959c-110dc60d7e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:08.789361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd978a-a050-440d-959c-110dc60d7e2e service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:08.789641 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd978a-a050-440d-959c-110dc60d7e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:08.789909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd978a-a050-440d-959c-110dc60d7e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:08.799453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd978a-a050-440d-959c-110dc60d7e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:08.800927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd978a-a050-440d-959c-110dc60d7e2e service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:08.810336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1332/2668] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:09.241154 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa699b-9e31-482a-9e4f-46840995af01 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:09.242677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-77aa699b-9e31-482a-9e4f-46840995af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:09.255626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa699b-9e31-482a-9e4f-46840995af01 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:09.260231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:09.260859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.261136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.261407 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] PATH_INFO: `/v3/projects/f6ca8ee07fe640d88546a4dfc09a08d9` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:09.262551 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ca8ee07fe640d88546a4dfc09a08d9)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:09.264492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:09.348643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ca8ee07fe640d88546a4dfc09a08d9', 'request_id': 'req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:09.355130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ca8ee07fe640d88546a4dfc09a08d9', 'request_id': 'req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:09.433155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ca8ee07fe640d88546a4dfc09a08d9', 'request_id': 'req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:09.443943 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] The token cache is being invalidate because project f6ca8ee07fe640d88546a4dfc09a08d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:32:09.444234 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:09.444868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-14e1bd3d-6349-4f88-a476-b1ac711a5f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:09.446384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1337/2669] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:32:09 2022] DELETE /identity/v3/projects/f6ca8ee07fe640d88546a4dfc09a08d9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:09.451356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054cf27-c502-4cea-b20b-6d8781527045 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.451875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054cf27-c502-4cea-b20b-6d8781527045 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.451992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054cf27-c502-4cea-b20b-6d8781527045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:09.472046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e054cf27-c502-4cea-b20b-6d8781527045 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:09.473249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e054cf27-c502-4cea-b20b-6d8781527045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:09.490395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:09.536064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1333/2670] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:32:09.544014 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c2ff0-d1e8-4209-ab0b-56acb1416855 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.544328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c2ff0-d1e8-4209-ab0b-56acb1416855 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.544607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c2ff0-d1e8-4209-ab0b-56acb1416855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:09.564915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f59c2ff0-d1e8-4209-ab0b-56acb1416855 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:09.566117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f59c2ff0-d1e8-4209-ab0b-56acb1416855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:09.574293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1338/2671] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:09.580759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054cf27-c502-4cea-b20b-6d8781527045 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:09.588083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054cf27-c502-4cea-b20b-6d8781527045 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:09.593103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:09.593713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.594202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.594344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:09.594960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:32:09.595291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:09.598479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:09.612175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1334/2672] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:32:09.616596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c2ff0-d1e8-4209-ab0b-56acb1416855 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:09.626118 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c2ff0-d1e8-4209-ab0b-56acb1416855 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:09.631028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab62039-59e5-427d-9667-fc5a8e7a28cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:09.631729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab62039-59e5-427d-9667-fc5a8e7a28cb admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.632082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab62039-59e5-427d-9667-fc5a8e7a28cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.632424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab62039-59e5-427d-9667-fc5a8e7a28cb admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:09.633133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab62039-59e5-427d-9667-fc5a8e7a28cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:09.635409 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab62039-59e5-427d-9667-fc5a8e7a28cb admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:09.750091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1339/2673] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:32:09 2022] POST /identity/v3/projects => generated 359 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:32:09.758355 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:09.769288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a293c-a191-4205-a0a6-fbf47ecdf553 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:09.824524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cddf4-a8b0-4432-9c1f-7fcfb41997ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:09.825740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cddf4-a8b0-4432-9c1f-7fcfb41997ab admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.826026 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cddf4-a8b0-4432-9c1f-7fcfb41997ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.826337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cddf4-a8b0-4432-9c1f-7fcfb41997ab admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:09.826980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cddf4-a8b0-4432-9c1f-7fcfb41997ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:09.828890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cddf4-a8b0-4432-9c1f-7fcfb41997ab admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:09.901860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1335/2674] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:32:09 2022] POST /identity/v3/users => generated 280 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:32:09.906008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab62039-59e5-427d-9667-fc5a8e7a28cb admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:09.917022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab62039-59e5-427d-9667-fc5a8e7a28cb admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:09.921688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eee0d8-e9b9-4078-a38b-50f22a3560dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:09.922325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eee0d8-e9b9-4078-a38b-50f22a3560dd admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.922607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eee0d8-e9b9-4078-a38b-50f22a3560dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.922880 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eee0d8-e9b9-4078-a38b-50f22a3560dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:09.923524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eee0d8-e9b9-4078-a38b-50f22a3560dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:09.925106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eee0d8-e9b9-4078-a38b-50f22a3560dd admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:09.935104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1340/2675] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:32:09.939179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cddf4-a8b0-4432-9c1f-7fcfb41997ab admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:09.949307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cddf4-a8b0-4432-9c1f-7fcfb41997ab admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:09.953845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7b457-8384-43b5-8538-52957c1a27a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:09.954548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7b457-8384-43b5-8538-52957c1a27a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.955357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7b457-8384-43b5-8538-52957c1a27a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.955628 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7b457-8384-43b5-8538-52957c1a27a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:09.956259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7b457-8384-43b5-8538-52957c1a27a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:09.957830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7b457-8384-43b5-8538-52957c1a27a7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:09.968339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1336/2676] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:32:09.973158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eee0d8-e9b9-4078-a38b-50f22a3560dd admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:09.983032 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eee0d8-e9b9-4078-a38b-50f22a3560dd admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:09.987785 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:09.988434 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:09.988712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:09.988985 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] PATH_INFO: `/v3/projects/a66f06fb12ea444dac802c04b418fd1a/users/d8c8113d30e54a9f8cd9744114010721/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:10.003503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66f06fb12ea444dac802c04b418fd1a, user_id=d8c8113d30e54a9f8cd9744114010721, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:10.006296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:10.062284 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1341/2677] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:32:09 2022] PUT /identity/v3/projects/a66f06fb12ea444dac802c04b418fd1a/users/d8c8113d30e54a9f8cd9744114010721/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:10.073354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9276dc-f174-453b-89e1-92494fca2d8c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:10.073643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9276dc-f174-453b-89e1-92494fca2d8c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:10.073918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9276dc-f174-453b-89e1-92494fca2d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:10.103789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-ec9276dc-f174-453b-89e1-92494fca2d8c None None] MFA Rules not processed for user `d8c8113d30e54a9f8cd9744114010721`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:10.105016 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-ec9276dc-f174-453b-89e1-92494fca2d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:10.173669 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1337/2678] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:32:10.210509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:10.212255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:10.234906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:10.272866 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7569e97-00b8-4748-8592-2e8f957b23b5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:10.276589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351667a-5a38-4e26-9836-8266e6c784f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:10.277193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351667a-5a38-4e26-9836-8266e6c784f1 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:10.277474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351667a-5a38-4e26-9836-8266e6c784f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:10.277782 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351667a-5a38-4e26-9836-8266e6c784f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:10.286537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351667a-5a38-4e26-9836-8266e6c784f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:10.287991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351667a-5a38-4e26-9836-8266e6c784f1 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:10.296840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1342/2679] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:10.312938 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9276dc-f174-453b-89e1-92494fca2d8c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:10.333119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:10.377226 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9276dc-f174-453b-89e1-92494fca2d8c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:10.382690 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:10.383423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:10.383799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:10.384139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] PATH_INFO: `/v3/users/d8c8113d30e54a9f8cd9744114010721` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:10.385413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c8113d30e54a9f8cd9744114010721)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:10.387667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:11.496134 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c8113d30e54a9f8cd9744114010721'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:11.593517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c8113d30e54a9f8cd9744114010721'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:11.594644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c8113d30e54a9f8cd9744114010721'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:11.595281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c8113d30e54a9f8cd9744114010721'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:11.603687 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c8113d30e54a9f8cd9744114010721'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:11.621899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1338/2680] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:32:10 2022] DELETE /identity/v3/users/d8c8113d30e54a9f8cd9744114010721 => generated 0 bytes in 1309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:11.626494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e77baf-7985-4251-86eb-18b55880da76 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:11.627035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e77baf-7985-4251-86eb-18b55880da76 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:11.627035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e77baf-7985-4251-86eb-18b55880da76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:11.646676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-f0e77baf-7985-4251-86eb-18b55880da76 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:11.647884 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f0e77baf-7985-4251-86eb-18b55880da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:11.708116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1343/2681] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:11.717890 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:11.722338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:11.778263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68e89-5324-4f75-8781-fb885466fb6f admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:11.781995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9c217-887f-4c2d-88a5-2a148d1b5dbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:11.782645 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9c217-887f-4c2d-88a5-2a148d1b5dbc service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:11.783480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9c217-887f-4c2d-88a5-2a148d1b5dbc service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:11.785396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9c217-887f-4c2d-88a5-2a148d1b5dbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:11.793987 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9c217-887f-4c2d-88a5-2a148d1b5dbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:11.795460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9c217-887f-4c2d-88a5-2a148d1b5dbc service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:11.806726 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1339/2682] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:12.258333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e77baf-7985-4251-86eb-18b55880da76 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:12.259779 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f0e77baf-7985-4251-86eb-18b55880da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:12.274682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e77baf-7985-4251-86eb-18b55880da76 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:12.279687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:12.280340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:12.280612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:12.280889 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] PATH_INFO: `/v3/projects/a66f06fb12ea444dac802c04b418fd1a` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:12.282850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66f06fb12ea444dac802c04b418fd1a)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:12.285449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:12.393919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66f06fb12ea444dac802c04b418fd1a', 'request_id': 'req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:12.401784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66f06fb12ea444dac802c04b418fd1a', 'request_id': 'req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:12.450674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66f06fb12ea444dac802c04b418fd1a', 'request_id': 'req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:12.465994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] The token cache is being invalidate because project a66f06fb12ea444dac802c04b418fd1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:32:12.466758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:12.467566 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a65cb2f2-a5e3-40e5-b93c-aa20b559903b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:12.469850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1344/2683] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:32:12 2022] DELETE /identity/v3/projects/a66f06fb12ea444dac802c04b418fd1a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:12.476177 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd562-86cf-4cf9-a1d6-22910108f95f None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:12.476470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd562-86cf-4cf9-a1d6-22910108f95f None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:12.476734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd562-86cf-4cf9-a1d6-22910108f95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:12.496031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5f4fd562-86cf-4cf9-a1d6-22910108f95f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:12.497263 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5f4fd562-86cf-4cf9-a1d6-22910108f95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:12.535318 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:12.561371 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1340/2684] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:12.571600 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108548-0c8a-48c5-89af-555c88758f15 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:12.571878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108548-0c8a-48c5-89af-555c88758f15 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:12.572146 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04108548-0c8a-48c5-89af-555c88758f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:12.593657 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-04108548-0c8a-48c5-89af-555c88758f15 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:12.595636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-04108548-0c8a-48c5-89af-555c88758f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:12.604277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1345/2685] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:12.609171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd562-86cf-4cf9-a1d6-22910108f95f None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:12.620307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd562-86cf-4cf9-a1d6-22910108f95f None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:12.625090 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:12.628397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:12.628397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:12.628397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:12.628397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:32:12.628397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:12.629321 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:12.643520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1341/2686] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:32:12.648291 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04108548-0c8a-48c5-89af-555c88758f15 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:12.659506 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04108548-0c8a-48c5-89af-555c88758f15 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:12.664357 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a660aa5-f2e1-4f44-a832-3e7f20e7f55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:12.665192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a660aa5-f2e1-4f44-a832-3e7f20e7f55b admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:12.665650 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a660aa5-f2e1-4f44-a832-3e7f20e7f55b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:12.666111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a660aa5-f2e1-4f44-a832-3e7f20e7f55b admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:12.666924 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a660aa5-f2e1-4f44-a832-3e7f20e7f55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:12.669705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a660aa5-f2e1-4f44-a832-3e7f20e7f55b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:12.789561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1346/2687] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:32:12 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:32:12.793758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:12.804580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1cabc-773e-43dd-a947-fb091d5f4fdf admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:12.878428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6f84b-5f7e-4065-9d6f-6dd6b3e169e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:12.879051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6f84b-5f7e-4065-9d6f-6dd6b3e169e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:12.879449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6f84b-5f7e-4065-9d6f-6dd6b3e169e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:12.879619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6f84b-5f7e-4065-9d6f-6dd6b3e169e0 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:12.880255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6f84b-5f7e-4065-9d6f-6dd6b3e169e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:12.882095 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6f84b-5f7e-4065-9d6f-6dd6b3e169e0 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:12.966510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1342/2688] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:32:12 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:32:12.970763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a660aa5-f2e1-4f44-a832-3e7f20e7f55b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:12.981549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a660aa5-f2e1-4f44-a832-3e7f20e7f55b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:12.988757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb606d-d0aa-421a-a1ae-f668b457086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:12.989455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb606d-d0aa-421a-a1ae-f668b457086e admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:12.989739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb606d-d0aa-421a-a1ae-f668b457086e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:12.990016 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb606d-d0aa-421a-a1ae-f668b457086e admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:12.990769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb606d-d0aa-421a-a1ae-f668b457086e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:12.994374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb606d-d0aa-421a-a1ae-f668b457086e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:13.008945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1347/2689] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:32:13.013079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6f84b-5f7e-4065-9d6f-6dd6b3e169e0 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:13.024406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6f84b-5f7e-4065-9d6f-6dd6b3e169e0 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:13.030241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1a9d0-46da-4ed9-b79f-e9297243c3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:13.031038 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1a9d0-46da-4ed9-b79f-e9297243c3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:13.031388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1a9d0-46da-4ed9-b79f-e9297243c3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:13.031729 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1a9d0-46da-4ed9-b79f-e9297243c3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:13.032439 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1a9d0-46da-4ed9-b79f-e9297243c3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:13.034920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1a9d0-46da-4ed9-b79f-e9297243c3f2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:13.047477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1343/2690] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:32:13.052168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb606d-d0aa-421a-a1ae-f668b457086e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:13.063133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb606d-d0aa-421a-a1ae-f668b457086e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:13.068771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:13.069379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:13.069731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:13.070102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] PATH_INFO: `/v3/projects/bc25b2c8f8244d8abe0e7c45d6f81af8/users/2245602676d540448cc891ea9c736792/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:13.072786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:13.086474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc25b2c8f8244d8abe0e7c45d6f81af8, user_id=2245602676d540448cc891ea9c736792, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:13.089160 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:13.148869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1348/2691] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:32:13 2022] PUT /identity/v3/projects/bc25b2c8f8244d8abe0e7c45d6f81af8/users/2245602676d540448cc891ea9c736792/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:32:13.159822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6785b3-01fe-4e49-bc1c-81ad2bc75b83 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:13.160106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6785b3-01fe-4e49-bc1c-81ad2bc75b83 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:13.160381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6785b3-01fe-4e49-bc1c-81ad2bc75b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:13.211516 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0f6785b3-01fe-4e49-bc1c-81ad2bc75b83 None None] MFA Rules not processed for user `2245602676d540448cc891ea9c736792`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:13.212865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0f6785b3-01fe-4e49-bc1c-81ad2bc75b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:13.290101 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1344/2692] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:13.305776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:13.308486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:13.382039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a751b7d-9625-4575-bf26-86f714ae254e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:13.385736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:13.386389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:13.386667 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:13.386941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:13.396279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:13.397706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:13.407697 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1349/2693] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:15.108058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f3fcd-660d-4b53-afb6-cfc7a9be0ac4 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:15.117039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f3fcd-660d-4b53-afb6-cfc7a9be0ac4 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:15.117039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f3fcd-660d-4b53-afb6-cfc7a9be0ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:15.138627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-d95f3fcd-660d-4b53-afb6-cfc7a9be0ac4 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:15.140137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d95f3fcd-660d-4b53-afb6-cfc7a9be0ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:15.176004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:15.203520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1345/2694] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 14:32:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:15.217216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:15.218717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:15.306026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0805f7-e905-4d36-a51c-a958e8349a64 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:15.312704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873757d-fca9-4514-9fd8-0a7a2a18b238 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:15.313328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873757d-fca9-4514-9fd8-0a7a2a18b238 service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:15.313613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873757d-fca9-4514-9fd8-0a7a2a18b238 service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:15.313891 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873757d-fca9-4514-9fd8-0a7a2a18b238 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:15.323738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873757d-fca9-4514-9fd8-0a7a2a18b238 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:15.325353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873757d-fca9-4514-9fd8-0a7a2a18b238 service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:15.335602 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1350/2695] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:32:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:32:25.330636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:25.558983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=9 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:37.779395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f3fcd-660d-4b53-afb6-cfc7a9be0ac4 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:37.843290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f3fcd-660d-4b53-afb6-cfc7a9be0ac4 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:37.848205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:37.848845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:37.849127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:37.849401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] PATH_INFO: `/v3/users/2245602676d540448cc891ea9c736792` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:37.850592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2245602676d540448cc891ea9c736792)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:37.853127 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:38.064858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2245602676d540448cc891ea9c736792'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:38.170876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2245602676d540448cc891ea9c736792'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:38.171513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2245602676d540448cc891ea9c736792'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:38.172130 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2245602676d540448cc891ea9c736792'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:38.180151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2245602676d540448cc891ea9c736792'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:38.202356 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1346/2696] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:32:37 2022] DELETE /identity/v3/users/2245602676d540448cc891ea9c736792 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:38.204323 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d484-73fc-4cb5-bfcf-5b29a582bf72 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:38.204811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d484-73fc-4cb5-bfcf-5b29a582bf72 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:38.204940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d484-73fc-4cb5-bfcf-5b29a582bf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:38.225177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-82e3d484-73fc-4cb5-bfcf-5b29a582bf72 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:38.226441 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-82e3d484-73fc-4cb5-bfcf-5b29a582bf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:38.286429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1351/2697] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:38.296633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:38.298060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:38.358004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a1c0a-04f0-46b1-a454-93498c21834d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:38.361598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89475c5-3b65-4365-8f47-f0a5dacc7d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:38.362244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89475c5-3b65-4365-8f47-f0a5dacc7d12 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:38.362527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89475c5-3b65-4365-8f47-f0a5dacc7d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:38.362798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89475c5-3b65-4365-8f47-f0a5dacc7d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:38.372810 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89475c5-3b65-4365-8f47-f0a5dacc7d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:38.374283 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89475c5-3b65-4365-8f47-f0a5dacc7d12 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:38.384369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1347/2698] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:38.835121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3d484-73fc-4cb5-bfcf-5b29a582bf72 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:38.836868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-82e3d484-73fc-4cb5-bfcf-5b29a582bf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:38.849108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3d484-73fc-4cb5-bfcf-5b29a582bf72 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:38.854017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:38.855237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:38.855237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:38.855502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] PATH_INFO: `/v3/projects/bc25b2c8f8244d8abe0e7c45d6f81af8` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:38.856474 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc25b2c8f8244d8abe0e7c45d6f81af8)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:38.858427 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:38.961260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc25b2c8f8244d8abe0e7c45d6f81af8', 'request_id': 'req-1225db3f-9727-4760-be52-5ed0d447e42c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:38.967786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc25b2c8f8244d8abe0e7c45d6f81af8', 'request_id': 'req-1225db3f-9727-4760-be52-5ed0d447e42c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:39.039399 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc25b2c8f8244d8abe0e7c45d6f81af8', 'request_id': 'req-1225db3f-9727-4760-be52-5ed0d447e42c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:39.050844 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] The token cache is being invalidate because project bc25b2c8f8244d8abe0e7c45d6f81af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:32:39.052823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:39.052823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1225db3f-9727-4760-be52-5ed0d447e42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:39.053393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1352/2699] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:32:38 2022] DELETE /identity/v3/projects/bc25b2c8f8244d8abe0e7c45d6f81af8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:39.058821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bc076-18ae-42e4-967c-56da35e57914 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.059364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bc076-18ae-42e4-967c-56da35e57914 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.059364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bc076-18ae-42e4-967c-56da35e57914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.080055 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-504bc076-18ae-42e4-967c-56da35e57914 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:39.081205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-504bc076-18ae-42e4-967c-56da35e57914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:39.097961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:39.142011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1348/2700] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:39.150816 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf01d-a8fd-4901-ad23-a3a9f1ed326f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.151104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf01d-a8fd-4901-ad23-a3a9f1ed326f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.151380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf01d-a8fd-4901-ad23-a3a9f1ed326f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.172342 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-17caf01d-a8fd-4901-ad23-a3a9f1ed326f None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:39.173562 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-17caf01d-a8fd-4901-ad23-a3a9f1ed326f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:39.182192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1353/2701] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:39.186632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bc076-18ae-42e4-967c-56da35e57914 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:39.196579 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bc076-18ae-42e4-967c-56da35e57914 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:39.201921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:39.202563 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.202839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.203111 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.203759 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:32:39.204057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:39.205647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:39.217543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1349/2702] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:32:39.222236 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caf01d-a8fd-4901-ad23-a3a9f1ed326f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:39.232062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caf01d-a8fd-4901-ad23-a3a9f1ed326f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:39.236787 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbd3e2-c88b-41aa-9308-eab72445882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:39.237381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd3e2-c88b-41aa-9308-eab72445882c admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.237659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd3e2-c88b-41aa-9308-eab72445882c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.238017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd3e2-c88b-41aa-9308-eab72445882c admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.238682 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbd3e2-c88b-41aa-9308-eab72445882c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:39.240573 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbd3e2-c88b-41aa-9308-eab72445882c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:39.332616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1354/2703] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:32:39 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:32:39.336506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:39.346585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54ebd8-7331-45a1-be49-f456717fd46e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:39.406681 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb4f80-c5f6-4208-bf9d-1cd195307688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:39.407293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb4f80-c5f6-4208-bf9d-1cd195307688 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.407578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb4f80-c5f6-4208-bf9d-1cd195307688 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.407854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb4f80-c5f6-4208-bf9d-1cd195307688 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.408481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb4f80-c5f6-4208-bf9d-1cd195307688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:39.410426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb4f80-c5f6-4208-bf9d-1cd195307688 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:39.513527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1350/2704] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:32:39 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:32:39.517988 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbd3e2-c88b-41aa-9308-eab72445882c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:39.528836 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbd3e2-c88b-41aa-9308-eab72445882c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:39.534439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e22f0e-df04-4fad-a368-a069733e46fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:39.535055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e22f0e-df04-4fad-a368-a069733e46fa admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.535335 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e22f0e-df04-4fad-a368-a069733e46fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.535606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e22f0e-df04-4fad-a368-a069733e46fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.536248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e22f0e-df04-4fad-a368-a069733e46fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:39.537842 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e22f0e-df04-4fad-a368-a069733e46fa admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:39.548843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1355/2705] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:32:39.553007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb4f80-c5f6-4208-bf9d-1cd195307688 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:39.564600 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb4f80-c5f6-4208-bf9d-1cd195307688 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:39.569445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a53d8f-7a8b-44da-ac4a-950201a43358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:39.570040 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a53d8f-7a8b-44da-ac4a-950201a43358 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.570357 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a53d8f-7a8b-44da-ac4a-950201a43358 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.570632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a53d8f-7a8b-44da-ac4a-950201a43358 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.571303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a53d8f-7a8b-44da-ac4a-950201a43358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:39.572910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a53d8f-7a8b-44da-ac4a-950201a43358 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:39.583873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1351/2706] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:32:39.588676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e22f0e-df04-4fad-a368-a069733e46fa admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:39.598938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e22f0e-df04-4fad-a368-a069733e46fa admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:39.604660 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:39.605309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.605590 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.605865 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] PATH_INFO: `/v3/projects/1928283cadf9477489b8046599331d95/users/5719fbb0b9da439fb24e7c67f4f25f1d/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.621381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1928283cadf9477489b8046599331d95, user_id=5719fbb0b9da439fb24e7c67f4f25f1d, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:39.624079 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:39.702315 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1356/2707] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:32:39 2022] PUT /identity/v3/projects/1928283cadf9477489b8046599331d95/users/5719fbb0b9da439fb24e7c67f4f25f1d/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:39.773171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90558f8b-4eb9-4c3d-adc2-5d9adf130b08 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.773459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90558f8b-4eb9-4c3d-adc2-5d9adf130b08 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.773736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90558f8b-4eb9-4c3d-adc2-5d9adf130b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.801683 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-90558f8b-4eb9-4c3d-adc2-5d9adf130b08 None None] MFA Rules not processed for user `5719fbb0b9da439fb24e7c67f4f25f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:39.802934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-90558f8b-4eb9-4c3d-adc2-5d9adf130b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:39.866550 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1352/2708] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:39.877218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:39.878982 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:39.902881 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:39.938517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298cb6-49cb-4248-b2ed-9f0f710b0308 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:39.942346 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a63f5a-cbfa-472f-955d-98ee5b7ed1ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:39.942953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a63f5a-cbfa-472f-955d-98ee5b7ed1ac service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:39.943231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a63f5a-cbfa-472f-955d-98ee5b7ed1ac service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:39.943500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a63f5a-cbfa-472f-955d-98ee5b7ed1ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:39.952919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a63f5a-cbfa-472f-955d-98ee5b7ed1ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:39.954378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a63f5a-cbfa-472f-955d-98ee5b7ed1ac service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:39.963901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1357/2709] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:43.952793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90558f8b-4eb9-4c3d-adc2-5d9adf130b08 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:43.972297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:44.015330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90558f8b-4eb9-4c3d-adc2-5d9adf130b08 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:44.020161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:44.020856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:44.021135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:44.021407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] PATH_INFO: `/v3/users/5719fbb0b9da439fb24e7c67f4f25f1d` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:44.022685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5719fbb0b9da439fb24e7c67f4f25f1d)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:44.025353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:44.305430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5719fbb0b9da439fb24e7c67f4f25f1d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:44.405752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5719fbb0b9da439fb24e7c67f4f25f1d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:44.406485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5719fbb0b9da439fb24e7c67f4f25f1d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:44.407153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5719fbb0b9da439fb24e7c67f4f25f1d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:44.414859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5719fbb0b9da439fb24e7c67f4f25f1d'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:44.432862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1353/2710] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:32:43 2022] DELETE /identity/v3/users/5719fbb0b9da439fb24e7c67f4f25f1d => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:44.437505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5ac0-c952-4717-8eab-dd3bf4ce4952 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:44.437987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5ac0-c952-4717-8eab-dd3bf4ce4952 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:44.438111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5ac0-c952-4717-8eab-dd3bf4ce4952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:44.457115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e6ad5ac0-c952-4717-8eab-dd3bf4ce4952 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:32:44.460307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e6ad5ac0-c952-4717-8eab-dd3bf4ce4952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:44.516874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1358/2711] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:32:44.527627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:44.529049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:44.586475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae1dc9-5766-4fb6-9729-510a73af197b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:44.590816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:44.591430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:44.591713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:44.591988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:44.601582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:44.603042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:44.612733 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1354/2712] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:45.084546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad5ac0-c952-4717-8eab-dd3bf4ce4952 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:45.086290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e6ad5ac0-c952-4717-8eab-dd3bf4ce4952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:45.100186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad5ac0-c952-4717-8eab-dd3bf4ce4952 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:45.105914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:45.106611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:45.106902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:45.107176 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] PATH_INFO: `/v3/projects/1928283cadf9477489b8046599331d95` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:45.108372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1928283cadf9477489b8046599331d95)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:45.110317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:45.268420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1928283cadf9477489b8046599331d95', 'request_id': 'req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:45.274486 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1928283cadf9477489b8046599331d95', 'request_id': 'req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:45.347688 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1928283cadf9477489b8046599331d95', 'request_id': 'req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:45.359077 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] The token cache is being invalidate because project 1928283cadf9477489b8046599331d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:32:45.359370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:45.359999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:32:45.361468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1359/2713] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:32:45 2022] DELETE /identity/v3/projects/1928283cadf9477489b8046599331d95 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:32:54.608262 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:55.355960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:58.280918 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:58.282380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:58.339732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f4c64-90a4-4efa-ab3e-dadf0c87e8dc service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:58.389011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:58.389624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:58.389913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:58.390214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:58.392238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:58.461332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:58.463027 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:58.474835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1355/2714] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:32:59.658714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:32:59.660189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:59.717075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcbab2-2d11-4b4c-a34d-26fdea2b86d1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:32:59.721370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1907ae4-ac8c-4bb5-ad41-35e707ec52e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:32:59.721975 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907ae4-ac8c-4bb5-ad41-35e707ec52e5 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:32:59.722288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907ae4-ac8c-4bb5-ad41-35e707ec52e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:32:59.722560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907ae4-ac8c-4bb5-ad41-35e707ec52e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:32:59.724370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a1907ae4-ac8c-4bb5-ad41-35e707ec52e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:32:59.738554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1907ae4-ac8c-4bb5-ad41-35e707ec52e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:32:59.740094 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1907ae4-ac8c-4bb5-ad41-35e707ec52e5 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:32:59.751152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1360/2715] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:32:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:33:08.470341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=74 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:33:09.746036 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=26 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:33:16.783727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:33:16.785165 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:33:16.903990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab52d-96b8-42d5-9519-7037b4518390 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:33:16.909404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:33:16.910031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:33:16.910319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:33:16.910595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:33:16.912384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:33:16.925458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:33:16.927064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:33:16.937023 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1356/2716] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:33:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:33:26.932710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=52 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:00.728626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3da53-57d7-4dcd-95bd-6e5781a310d1 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:34:00.729132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3da53-57d7-4dcd-95bd-6e5781a310d1 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:34:00.729333 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3da53-57d7-4dcd-95bd-6e5781a310d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:34:00.749100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-35e3da53-57d7-4dcd-95bd-6e5781a310d1 None None] MFA Rules not processed for user `357ebb86fd664e08b15bf440e2a61560`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:34:00.750391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-35e3da53-57d7-4dcd-95bd-6e5781a310d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:34:00.821946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1361/2717] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:34:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:34:10.820137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=33 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:40.188999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:34:40.190440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:34:40.211192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bc231-3e63-4f03-b649-620dc74f9828 service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:34:40.215310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:34:40.215906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:34:40.216189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:34:40.216461 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:34:40.218167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:34:40.293496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:34:40.294952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:34:40.304949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1357/2718] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:34:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:34:50.300709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=38 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:21.594849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3da53-57d7-4dcd-95bd-6e5781a310d1 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:21.596962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-35e3da53-57d7-4dcd-95bd-6e5781a310d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:21.650681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3da53-57d7-4dcd-95bd-6e5781a310d1 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:21.658768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:21.658768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:21.658768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:21.659244 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:21.661324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:21.676811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:21.678648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:21.691596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1362/2719] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:23.625102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:23.638031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d572e-1505-4ae5-8de7-0874276a08b8 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:23.643753 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:23.644418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:23.644701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:23.644977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:23.647024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:23.657807 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:23.660089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:23.669245 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1358/2720] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:28.602018 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:28.611478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dbd87-e2ce-488d-a01b-188bb022f8a7 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:28.615328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:28.616508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:28.616784 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:28.617053 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:28.618928 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:28.684691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:28.686151 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:28.695993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1363/2721] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:31.994793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:31.996255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:32.011138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5c8a1-e4c0-46f1-8430-202638b302ac service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:32.016460 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:32.017064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:32.017343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:32.017619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:32.019418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:32.032087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:32.033666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:32.044543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1359/2722] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:32.139552 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:32.141022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:32.154957 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a258e-28fd-4af5-8312-523d4a40eea1 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:32.159673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:32.161672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:32.161672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:32.161672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] PATH_INFO: `/v3/users/357ebb86fd664e08b15bf440e2a61560` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:32.162008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=357ebb86fd664e08b15bf440e2a61560)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:32.164065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:32.362776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357ebb86fd664e08b15bf440e2a61560'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.433804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357ebb86fd664e08b15bf440e2a61560'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.434450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357ebb86fd664e08b15bf440e2a61560'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.447294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357ebb86fd664e08b15bf440e2a61560'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.454361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c30a6dc2-713c-4c4a-b0df-330d9934e478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357ebb86fd664e08b15bf440e2a61560'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.461101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1364/2723] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:35:32 2022] DELETE /identity/v3/users/357ebb86fd664e08b15bf440e2a61560 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:32.681625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:32.693458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e98b7f-35b4-41e8-bb1f-f4edb6184b69 service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:32.753448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:32.754005 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:32.754331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:32.754610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] PATH_INFO: `/v3/projects/b21f666643b74b5cba0b7597ae1aa0cd` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:32.756270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21f666643b74b5cba0b7597ae1aa0cd)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:32.758204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:32.858044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21f666643b74b5cba0b7597ae1aa0cd', 'request_id': 'req-198770a1-dfcf-4757-8775-c8381a01f5bc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.864298 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21f666643b74b5cba0b7597ae1aa0cd', 'request_id': 'req-198770a1-dfcf-4757-8775-c8381a01f5bc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.925338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21f666643b74b5cba0b7597ae1aa0cd', 'request_id': 'req-198770a1-dfcf-4757-8775-c8381a01f5bc'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.937570 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] The token cache is being invalidate because project b21f666643b74b5cba0b7597ae1aa0cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:35:32.937863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.938533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-198770a1-dfcf-4757-8775-c8381a01f5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:32.940019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1360/2724] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:35:32 2022] DELETE /identity/v3/projects/b21f666643b74b5cba0b7597ae1aa0cd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:32.945417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6516b06-99da-4b63-9680-58a90ef09e82 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:32.945702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6516b06-99da-4b63-9680-58a90ef09e82 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:32.946074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6516b06-99da-4b63-9680-58a90ef09e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:32.966765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a6516b06-99da-4b63-9680-58a90ef09e82 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:32.967992 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a6516b06-99da-4b63-9680-58a90ef09e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:33.006293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:33.038707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1365/2725] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:33.046458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e12a3-5ca1-425b-95e0-5574d1dac500 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.046933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e12a3-5ca1-425b-95e0-5574d1dac500 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.047171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e12a3-5ca1-425b-95e0-5574d1dac500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.070202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-fe2e12a3-5ca1-425b-95e0-5574d1dac500 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:33.071534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fe2e12a3-5ca1-425b-95e0-5574d1dac500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:33.080448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1361/2726] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:33.086385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6516b06-99da-4b63-9680-58a90ef09e82 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:33.096430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6516b06-99da-4b63-9680-58a90ef09e82 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:33.101561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:33.102270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.102830 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.103334 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.104369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:35:33.104737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:33.106464 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:33.119310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1366/2727] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:35:33.123777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e12a3-5ca1-425b-95e0-5574d1dac500 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:33.133879 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e12a3-5ca1-425b-95e0-5574d1dac500 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:33.139224 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a7fea-ee9a-4e36-862d-f19209eabb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:33.139819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a7fea-ee9a-4e36-862d-f19209eabb0c admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.140096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a7fea-ee9a-4e36-862d-f19209eabb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.140369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a7fea-ee9a-4e36-862d-f19209eabb0c admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.141061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a7fea-ee9a-4e36-862d-f19209eabb0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:33.143979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a7fea-ee9a-4e36-862d-f19209eabb0c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:33.259813 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1362/2728] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:35:33 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:33.263911 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:33.274424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ab9b-2ac6-4b3c-8b14-4df942fc7351 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:33.338981 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff191d-8413-44ad-a6e9-6ce4d49ff7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:33.339591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff191d-8413-44ad-a6e9-6ce4d49ff7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.339868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff191d-8413-44ad-a6e9-6ce4d49ff7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.340169 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff191d-8413-44ad-a6e9-6ce4d49ff7f9 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.340806 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff191d-8413-44ad-a6e9-6ce4d49ff7f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:33.342926 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff191d-8413-44ad-a6e9-6ce4d49ff7f9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:33.451837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1367/2729] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:35:33 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:35:33.456083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a7fea-ee9a-4e36-862d-f19209eabb0c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:33.466518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a7fea-ee9a-4e36-862d-f19209eabb0c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:33.471179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a43a7-6fad-4be8-a488-6267a7db63b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:33.471783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a43a7-6fad-4be8-a488-6267a7db63b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.472066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a43a7-6fad-4be8-a488-6267a7db63b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.472341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a43a7-6fad-4be8-a488-6267a7db63b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.473396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a43a7-6fad-4be8-a488-6267a7db63b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:33.474750 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a43a7-6fad-4be8-a488-6267a7db63b8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:33.485093 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1363/2730] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:33.489198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff191d-8413-44ad-a6e9-6ce4d49ff7f9 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:33.499817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff191d-8413-44ad-a6e9-6ce4d49ff7f9 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:33.504495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae8163-94c8-4996-8ea3-27778f06de04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:33.505090 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8163-94c8-4996-8ea3-27778f06de04 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.505370 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8163-94c8-4996-8ea3-27778f06de04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.505642 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8163-94c8-4996-8ea3-27778f06de04 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.506883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae8163-94c8-4996-8ea3-27778f06de04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:33.508461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae8163-94c8-4996-8ea3-27778f06de04 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:33.519055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1368/2731] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:33.523912 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a43a7-6fad-4be8-a488-6267a7db63b8 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:33.534191 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a43a7-6fad-4be8-a488-6267a7db63b8 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:33.539483 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a1f51-f4a6-4437-ac9f-5a4296cb5a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:33.540143 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1f51-f4a6-4437-ac9f-5a4296cb5a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.541464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1f51-f4a6-4437-ac9f-5a4296cb5a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.541464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1f51-f4a6-4437-ac9f-5a4296cb5a3d admin admin] PATH_INFO: `/v3/projects/271ef4522e9c4baeabf1b4e891830699/users/ab2e658b61a04c0bb365d1136d043867/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.556173 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a1f51-f4a6-4437-ac9f-5a4296cb5a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=271ef4522e9c4baeabf1b4e891830699, user_id=ab2e658b61a04c0bb365d1136d043867, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:33.558868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a1f51-f4a6-4437-ac9f-5a4296cb5a3d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:33.630219 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:33.636325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1364/2732] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:33 2022] PUT /identity/v3/projects/271ef4522e9c4baeabf1b4e891830699/users/ab2e658b61a04c0bb365d1136d043867/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:33.700737 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ababfa-ff2e-4e3d-b9fe-ff2162fd55ff None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.701025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ababfa-ff2e-4e3d-b9fe-ff2162fd55ff None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.701299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ababfa-ff2e-4e3d-b9fe-ff2162fd55ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.728977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-a1ababfa-ff2e-4e3d-b9fe-ff2162fd55ff None None] MFA Rules not processed for user `ab2e658b61a04c0bb365d1136d043867`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:33.730263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a1ababfa-ff2e-4e3d-b9fe-ff2162fd55ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:33.794400 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1369/2733] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:33.798868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a1f51-f4a6-4437-ac9f-5a4296cb5a3d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:33.860120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a1f51-f4a6-4437-ac9f-5a4296cb5a3d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:33.865255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151fdd0-a022-4270-9572-4aa9a0213133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:33.865863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151fdd0-a022-4270-9572-4aa9a0213133 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:33.866158 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151fdd0-a022-4270-9572-4aa9a0213133 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:33.866450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151fdd0-a022-4270-9572-4aa9a0213133 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:33.867091 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151fdd0-a022-4270-9572-4aa9a0213133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:33.869017 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151fdd0-a022-4270-9572-4aa9a0213133 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:33.961104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1365/2734] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:35:33 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:33.965213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ababfa-ff2e-4e3d-b9fe-ff2162fd55ff None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:33.976209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ababfa-ff2e-4e3d-b9fe-ff2162fd55ff None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:34.005694 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:34.032468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981241d4-7865-4551-8799-57e6902664a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:34.033155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981241d4-7865-4551-8799-57e6902664a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:34.033436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981241d4-7865-4551-8799-57e6902664a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:34.033715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981241d4-7865-4551-8799-57e6902664a7 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:34.034643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981241d4-7865-4551-8799-57e6902664a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:34.037369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981241d4-7865-4551-8799-57e6902664a7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:34.126064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1370/2735] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:35:33 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:34.130115 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151fdd0-a022-4270-9572-4aa9a0213133 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:34.141271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151fdd0-a022-4270-9572-4aa9a0213133 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:34.145812 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc7c83-2d6a-4e3c-9b04-b2e6269f9628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:34.146719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc7c83-2d6a-4e3c-9b04-b2e6269f9628 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:34.147109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc7c83-2d6a-4e3c-9b04-b2e6269f9628 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:34.147266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc7c83-2d6a-4e3c-9b04-b2e6269f9628 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:34.147906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc7c83-2d6a-4e3c-9b04-b2e6269f9628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:34.149488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc7c83-2d6a-4e3c-9b04-b2e6269f9628 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:34.160052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1366/2736] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:34.164411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981241d4-7865-4551-8799-57e6902664a7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:34.174348 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981241d4-7865-4551-8799-57e6902664a7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:34.179967 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8c4e-c03b-4aa6-bbc0-a6a7bfb6e827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:34.180612 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8c4e-c03b-4aa6-bbc0-a6a7bfb6e827 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:34.180890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8c4e-c03b-4aa6-bbc0-a6a7bfb6e827 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:34.181163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8c4e-c03b-4aa6-bbc0-a6a7bfb6e827 admin admin] PATH_INFO: `/v3/projects/7072d6a33bc54ff0a9738ab4c5442336/users/d62d1de1ca224741ab145aa36a6f0135/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:34.196591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe8c4e-c03b-4aa6-bbc0-a6a7bfb6e827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7072d6a33bc54ff0a9738ab4c5442336, user_id=d62d1de1ca224741ab145aa36a6f0135, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:34.199299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe8c4e-c03b-4aa6-bbc0-a6a7bfb6e827 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:34.280512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1371/2737] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:34 2022] PUT /identity/v3/projects/7072d6a33bc54ff0a9738ab4c5442336/users/d62d1de1ca224741ab145aa36a6f0135/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:34.284754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc7c83-2d6a-4e3c-9b04-b2e6269f9628 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:34.295148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc7c83-2d6a-4e3c-9b04-b2e6269f9628 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:34.351964 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e4f5f-c6eb-4806-bd81-e91ee63c84b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:34.352582 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e4f5f-c6eb-4806-bd81-e91ee63c84b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:34.352865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e4f5f-c6eb-4806-bd81-e91ee63c84b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:34.353146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e4f5f-c6eb-4806-bd81-e91ee63c84b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:34.353784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e4f5f-c6eb-4806-bd81-e91ee63c84b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:34.355419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e4f5f-c6eb-4806-bd81-e91ee63c84b1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:34.366163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1367/2738] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:34.370776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8c4e-c03b-4aa6-bbc0-a6a7bfb6e827 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:34.383747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8c4e-c03b-4aa6-bbc0-a6a7bfb6e827 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:34.389132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:34.389817 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:34.390097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:34.390402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] PATH_INFO: `/v3/projects/7072d6a33bc54ff0a9738ab4c5442336/users/d62d1de1ca224741ab145aa36a6f0135/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:34.392423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7072d6a33bc54ff0a9738ab4c5442336, user_id=d62d1de1ca224741ab145aa36a6f0135, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:34.395074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:34.406546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1372/2739] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:34 2022] PUT /identity/v3/projects/7072d6a33bc54ff0a9738ab4c5442336/users/d62d1de1ca224741ab145aa36a6f0135/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:35:34.416999 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198255e-62ea-419b-9aae-408e87d92f54 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:34.417294 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198255e-62ea-419b-9aae-408e87d92f54 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:34.417571 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198255e-62ea-419b-9aae-408e87d92f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:34.446162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5198255e-62ea-419b-9aae-408e87d92f54 None None] MFA Rules not processed for user `d62d1de1ca224741ab145aa36a6f0135`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:34.447465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5198255e-62ea-419b-9aae-408e87d92f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:34.507989 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:34.517653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1368/2740] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:34.529561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:34.531287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:34.588575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70cd94-22db-415a-b96d-c0825b02f5db admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:34.592240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de08fe-6277-4cf2-9152-05524d076833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:34.592845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de08fe-6277-4cf2-9152-05524d076833 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:34.593123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de08fe-6277-4cf2-9152-05524d076833 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:34.593398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de08fe-6277-4cf2-9152-05524d076833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:34.603197 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de08fe-6277-4cf2-9152-05524d076833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:34.604626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de08fe-6277-4cf2-9152-05524d076833 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:34.613972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1373/2741] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:34.814491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198255e-62ea-419b-9aae-408e87d92f54 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:34.825024 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198255e-62ea-419b-9aae-408e87d92f54 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:34.828688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae8971-c1c5-4991-b164-ab862877605e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:34.831580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae8971-c1c5-4991-b164-ab862877605e service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:34.831580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae8971-c1c5-4991-b164-ab862877605e service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:34.831580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae8971-c1c5-4991-b164-ab862877605e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:34.898464 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae8971-c1c5-4991-b164-ab862877605e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:34.899818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae8971-c1c5-4991-b164-ab862877605e service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:34.908672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1369/2742] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:44.609773 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=55,keystone:INSERT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:44.904573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=30 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:46.490155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de08fe-6277-4cf2-9152-05524d076833 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:46.551029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de08fe-6277-4cf2-9152-05524d076833 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:46.555514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:46.562528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:46.562528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:46.562528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] PATH_INFO: `/v3/users/ab2e658b61a04c0bb365d1136d043867` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:46.562528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2e658b61a04c0bb365d1136d043867)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:46.562528 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:46.757366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2e658b61a04c0bb365d1136d043867'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:46.817143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2e658b61a04c0bb365d1136d043867'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:46.817756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2e658b61a04c0bb365d1136d043867'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:46.830504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2e658b61a04c0bb365d1136d043867'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:46.837598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2e658b61a04c0bb365d1136d043867'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:46.843460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1374/2743] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:35:46 2022] DELETE /identity/v3/users/ab2e658b61a04c0bb365d1136d043867 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:46.848132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d187b-ee11-4414-b858-480c609990f1 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:46.848609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d187b-ee11-4414-b858-480c609990f1 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:46.848711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d187b-ee11-4414-b858-480c609990f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:46.868581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-dd0d187b-ee11-4414-b858-480c609990f1 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:46.869795 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-dd0d187b-ee11-4414-b858-480c609990f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:46.930655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1370/2744] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:46.941372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:46.942914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:47.000269 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2e2d6-34e3-4a62-a50a-d86a5bbba1a7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:47.003930 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080761a-79af-4898-bde9-5f8836b4025b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:47.004539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080761a-79af-4898-bde9-5f8836b4025b service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:47.004812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080761a-79af-4898-bde9-5f8836b4025b service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:47.005081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080761a-79af-4898-bde9-5f8836b4025b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:47.013977 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080761a-79af-4898-bde9-5f8836b4025b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:47.015429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080761a-79af-4898-bde9-5f8836b4025b service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:47.025028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1375/2745] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:47.464323 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d187b-ee11-4414-b858-480c609990f1 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:47.465756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-dd0d187b-ee11-4414-b858-480c609990f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:47.478292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d187b-ee11-4414-b858-480c609990f1 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:47.483555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:47.484250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:47.484639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:47.484856 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] PATH_INFO: `/v3/projects/271ef4522e9c4baeabf1b4e891830699` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:47.486031 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] RBAC: Authorizing `identity:delete_project(project_id=271ef4522e9c4baeabf1b4e891830699)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:47.488107 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:47.628853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271ef4522e9c4baeabf1b4e891830699', 'request_id': 'req-14c845f2-68ef-46b9-8144-f8e53830793a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:47.635028 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271ef4522e9c4baeabf1b4e891830699', 'request_id': 'req-14c845f2-68ef-46b9-8144-f8e53830793a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:47.742862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271ef4522e9c4baeabf1b4e891830699', 'request_id': 'req-14c845f2-68ef-46b9-8144-f8e53830793a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:47.754327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] The token cache is being invalidate because project 271ef4522e9c4baeabf1b4e891830699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:35:47.754620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:47.755239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:47.756833 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1371/2746] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:35:47 2022] DELETE /identity/v3/projects/271ef4522e9c4baeabf1b4e891830699 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:47.760898 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080761a-79af-4898-bde9-5f8836b4025b service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:47.770608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080761a-79af-4898-bde9-5f8836b4025b service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:47.788828 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:47.828313 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:47.828939 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:47.829220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:47.829527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] PATH_INFO: `/v3/users/d62d1de1ca224741ab145aa36a6f0135` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:47.830683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62d1de1ca224741ab145aa36a6f0135)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:47.832651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:48.008237 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62d1de1ca224741ab145aa36a6f0135'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.073890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62d1de1ca224741ab145aa36a6f0135'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.074532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62d1de1ca224741ab145aa36a6f0135'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.086980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62d1de1ca224741ab145aa36a6f0135'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.094377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-576c49cd-1411-405f-be54-9a5d3448a4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62d1de1ca224741ab145aa36a6f0135'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.101267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1376/2747] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:35:47 2022] DELETE /identity/v3/users/d62d1de1ca224741ab145aa36a6f0135 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:48.527617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:48.529060 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:48.547544 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c845f2-68ef-46b9-8144-f8e53830793a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:48.605075 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:48.605719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:48.606149 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:48.606431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] PATH_INFO: `/v3/projects/7072d6a33bc54ff0a9738ab4c5442336` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:48.607580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7072d6a33bc54ff0a9738ab4c5442336)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:48.610041 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:48.704217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:48.720634 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7072d6a33bc54ff0a9738ab4c5442336', 'request_id': 'req-21b93d7d-3b74-4a76-92d7-99adacf43e1a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.726777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7072d6a33bc54ff0a9738ab4c5442336', 'request_id': 'req-21b93d7d-3b74-4a76-92d7-99adacf43e1a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.834342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7072d6a33bc54ff0a9738ab4c5442336', 'request_id': 'req-21b93d7d-3b74-4a76-92d7-99adacf43e1a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.846076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] The token cache is being invalidate because project 7072d6a33bc54ff0a9738ab4c5442336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:35:48.846625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.847871 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-21b93d7d-3b74-4a76-92d7-99adacf43e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:48.849380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1372/2748] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:35:48 2022] DELETE /identity/v3/projects/7072d6a33bc54ff0a9738ab4c5442336 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:48.855270 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c8c7b-2d1e-4b15-a9d4-6d42833c1cb8 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:48.855748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c8c7b-2d1e-4b15-a9d4-6d42833c1cb8 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:48.855848 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c8c7b-2d1e-4b15-a9d4-6d42833c1cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:48.876183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-153c8c7b-2d1e-4b15-a9d4-6d42833c1cb8 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:48.877359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-153c8c7b-2d1e-4b15-a9d4-6d42833c1cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:48.938534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1377/2749] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:35:48.947893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c321a4-e147-4a1f-ba4f-104e99a0bc57 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:48.948180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c321a4-e147-4a1f-ba4f-104e99a0bc57 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:48.948454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c321a4-e147-4a1f-ba4f-104e99a0bc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:48.969785 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-24c321a4-e147-4a1f-ba4f-104e99a0bc57 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:48.971025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-24c321a4-e147-4a1f-ba4f-104e99a0bc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:48.980583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1373/2750] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:48.985953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c8c7b-2d1e-4b15-a9d4-6d42833c1cb8 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:48.995833 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c8c7b-2d1e-4b15-a9d4-6d42833c1cb8 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:49.000405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:49.000997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.001276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.001546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.002223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:35:49.002527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:49.004312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:49.015677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1378/2751] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:35:49.020042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c321a4-e147-4a1f-ba4f-104e99a0bc57 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:49.029877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c321a4-e147-4a1f-ba4f-104e99a0bc57 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:49.035513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4be99-65ec-43de-b1d9-02f83b5a296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:49.036132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4be99-65ec-43de-b1d9-02f83b5a296c admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.036398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4be99-65ec-43de-b1d9-02f83b5a296c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.036674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4be99-65ec-43de-b1d9-02f83b5a296c admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.037320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4be99-65ec-43de-b1d9-02f83b5a296c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:49.039271 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4be99-65ec-43de-b1d9-02f83b5a296c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:49.132274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1374/2752] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:35:49 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:49.136318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:49.147108 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859085-2ba1-449e-88f8-e331cf9e1be1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:49.172966 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:49.203804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554e144-a049-4a88-93bf-6ac36c8c9b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:49.204425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e144-a049-4a88-93bf-6ac36c8c9b57 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.204703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e144-a049-4a88-93bf-6ac36c8c9b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.204993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554e144-a049-4a88-93bf-6ac36c8c9b57 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.205621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554e144-a049-4a88-93bf-6ac36c8c9b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:49.207575 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554e144-a049-4a88-93bf-6ac36c8c9b57 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:49.343165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1379/2753] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:35:49 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:49.347147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4be99-65ec-43de-b1d9-02f83b5a296c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:49.357620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4be99-65ec-43de-b1d9-02f83b5a296c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:49.362761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f537581-49b0-4782-aec3-c3e8dd2a616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:49.363376 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f537581-49b0-4782-aec3-c3e8dd2a616e admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.363655 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f537581-49b0-4782-aec3-c3e8dd2a616e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.363927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f537581-49b0-4782-aec3-c3e8dd2a616e admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.364590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f537581-49b0-4782-aec3-c3e8dd2a616e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:49.366201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f537581-49b0-4782-aec3-c3e8dd2a616e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:49.376673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1375/2754] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:49.380629 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554e144-a049-4a88-93bf-6ac36c8c9b57 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:49.391132 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554e144-a049-4a88-93bf-6ac36c8c9b57 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:49.395695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c351e-2039-439c-85fa-eb77a3ba25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:49.396288 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c351e-2039-439c-85fa-eb77a3ba25cf admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.396837 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c351e-2039-439c-85fa-eb77a3ba25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.397112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c351e-2039-439c-85fa-eb77a3ba25cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.397750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c351e-2039-439c-85fa-eb77a3ba25cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:49.399381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c351e-2039-439c-85fa-eb77a3ba25cf admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:49.410584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1380/2755] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:49.414666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f537581-49b0-4782-aec3-c3e8dd2a616e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:49.424700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f537581-49b0-4782-aec3-c3e8dd2a616e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:49.429378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b44b52-fbbc-4cb0-915c-ed606c492dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:49.430012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b44b52-fbbc-4cb0-915c-ed606c492dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.430334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b44b52-fbbc-4cb0-915c-ed606c492dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.430613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b44b52-fbbc-4cb0-915c-ed606c492dd8 admin admin] PATH_INFO: `/v3/projects/d5633c9cc15f42eba1d6d4049b5e326b/users/5b08841bbfab4a858f735c1ff17df1e4/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.445693 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b44b52-fbbc-4cb0-915c-ed606c492dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5633c9cc15f42eba1d6d4049b5e326b, user_id=5b08841bbfab4a858f735c1ff17df1e4, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:49.448385 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b44b52-fbbc-4cb0-915c-ed606c492dd8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:49.525568 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1376/2756] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:49 2022] PUT /identity/v3/projects/d5633c9cc15f42eba1d6d4049b5e326b/users/5b08841bbfab4a858f735c1ff17df1e4/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:49.535644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb4427-16d6-4e04-a14c-0b666420d7d0 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.535878 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb4427-16d6-4e04-a14c-0b666420d7d0 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.536239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb4427-16d6-4e04-a14c-0b666420d7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.563863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-50cb4427-16d6-4e04-a14c-0b666420d7d0 None None] MFA Rules not processed for user `5b08841bbfab4a858f735c1ff17df1e4`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:49.565074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-50cb4427-16d6-4e04-a14c-0b666420d7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:49.628627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1381/2757] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:49.633232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b44b52-fbbc-4cb0-915c-ed606c492dd8 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:49.695527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b44b52-fbbc-4cb0-915c-ed606c492dd8 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:49.700169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e9228-8f40-40dd-b64a-283b65296be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:49.700767 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e9228-8f40-40dd-b64a-283b65296be6 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.701044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e9228-8f40-40dd-b64a-283b65296be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.701315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e9228-8f40-40dd-b64a-283b65296be6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.702533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e9228-8f40-40dd-b64a-283b65296be6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:49.704421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e9228-8f40-40dd-b64a-283b65296be6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:49.863853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1377/2758] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:35:49 2022] POST /identity/v3/projects => generated 389 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:49.868068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb4427-16d6-4e04-a14c-0b666420d7d0 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:49.879363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb4427-16d6-4e04-a14c-0b666420d7d0 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:49.937905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491b671-1c8b-4c1c-b5d3-9eeacb4d779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:49.938851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b671-1c8b-4c1c-b5d3-9eeacb4d779f admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:49.939331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b671-1c8b-4c1c-b5d3-9eeacb4d779f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:49.939777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b671-1c8b-4c1c-b5d3-9eeacb4d779f admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:49.940808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491b671-1c8b-4c1c-b5d3-9eeacb4d779f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:49.942781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491b671-1c8b-4c1c-b5d3-9eeacb4d779f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:50.046182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1382/2759] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:35:49 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:50.050223 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e9228-8f40-40dd-b64a-283b65296be6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:50.060820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e9228-8f40-40dd-b64a-283b65296be6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:50.066036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387af554-1a60-4adb-9aa1-0a34c7980b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:50.066675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af554-1a60-4adb-9aa1-0a34c7980b55 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:50.066990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af554-1a60-4adb-9aa1-0a34c7980b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:50.067386 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af554-1a60-4adb-9aa1-0a34c7980b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:50.068032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387af554-1a60-4adb-9aa1-0a34c7980b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:50.069619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387af554-1a60-4adb-9aa1-0a34c7980b55 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:50.080349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1378/2760] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:50.084725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491b671-1c8b-4c1c-b5d3-9eeacb4d779f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:50.095044 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491b671-1c8b-4c1c-b5d3-9eeacb4d779f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:50.099942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4db03c-913b-4eb5-b170-58359f6b750b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:50.100613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db03c-913b-4eb5-b170-58359f6b750b admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:50.100892 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db03c-913b-4eb5-b170-58359f6b750b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:50.101158 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db03c-913b-4eb5-b170-58359f6b750b admin admin] PATH_INFO: `/v3/projects/f2057316b9324dc581dd447861c8349e/users/b0eddb08ae53408683598c1b6260f784/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:50.116478 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4db03c-913b-4eb5-b170-58359f6b750b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2057316b9324dc581dd447861c8349e, user_id=b0eddb08ae53408683598c1b6260f784, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:50.119150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4db03c-913b-4eb5-b170-58359f6b750b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:50.238162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1383/2761] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:50 2022] PUT /identity/v3/projects/f2057316b9324dc581dd447861c8349e/users/b0eddb08ae53408683598c1b6260f784/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:50.243276 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387af554-1a60-4adb-9aa1-0a34c7980b55 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:50.253867 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387af554-1a60-4adb-9aa1-0a34c7980b55 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:50.274940 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:50.314691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d9367-e34b-4132-865a-d403894cfdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:50.315222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d9367-e34b-4132-865a-d403894cfdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:50.315500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d9367-e34b-4132-865a-d403894cfdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:50.315788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d9367-e34b-4132-865a-d403894cfdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:50.316408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d9367-e34b-4132-865a-d403894cfdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:50.318025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d9367-e34b-4132-865a-d403894cfdb9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:50.328751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1379/2762] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:50.332916 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4db03c-913b-4eb5-b170-58359f6b750b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:50.338240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:50.344167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4db03c-913b-4eb5-b170-58359f6b750b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:50.348755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:50.350385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:50.350385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:50.350385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] PATH_INFO: `/v3/projects/f2057316b9324dc581dd447861c8349e/users/b0eddb08ae53408683598c1b6260f784/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:50.351826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2057316b9324dc581dd447861c8349e, user_id=b0eddb08ae53408683598c1b6260f784, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:50.354587 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:50.365220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1384/2763] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:50 2022] PUT /identity/v3/projects/f2057316b9324dc581dd447861c8349e/users/b0eddb08ae53408683598c1b6260f784/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:50.375499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172f0e0-de22-4ac5-8a62-e98ea49b7c7d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:50.375786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172f0e0-de22-4ac5-8a62-e98ea49b7c7d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:50.376063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172f0e0-de22-4ac5-8a62-e98ea49b7c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:50.404511 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f172f0e0-de22-4ac5-8a62-e98ea49b7c7d None None] MFA Rules not processed for user `b0eddb08ae53408683598c1b6260f784`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:50.407155 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f172f0e0-de22-4ac5-8a62-e98ea49b7c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:50.475197 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1380/2764] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:50 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:50.486527 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:50.488272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:50.567762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0fc3d-070b-4ded-8003-11a91952d4e1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:50.573043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a6ba2-6790-4474-af24-3e60af49fa23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:50.574103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a6ba2-6790-4474-af24-3e60af49fa23 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:50.574637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a6ba2-6790-4474-af24-3e60af49fa23 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:50.575126 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a6ba2-6790-4474-af24-3e60af49fa23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:50.587430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a6ba2-6790-4474-af24-3e60af49fa23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:50.588869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a6ba2-6790-4474-af24-3e60af49fa23 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:50.599326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1385/2765] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:50 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:50.735479 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172f0e0-de22-4ac5-8a62-e98ea49b7c7d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:50.798656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172f0e0-de22-4ac5-8a62-e98ea49b7c7d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:50.803290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:50.803919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:50.804201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:50.804478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] PATH_INFO: `/v3/users/5b08841bbfab4a858f735c1ff17df1e4` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:50.805609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b08841bbfab4a858f735c1ff17df1e4)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:50.807588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:51.026991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b08841bbfab4a858f735c1ff17df1e4'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.028696 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:51.097113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b08841bbfab4a858f735c1ff17df1e4'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.097731 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b08841bbfab4a858f735c1ff17df1e4'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.098395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b08841bbfab4a858f735c1ff17df1e4'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.105818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b08841bbfab4a858f735c1ff17df1e4'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.124981 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1381/2766] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:35:50 2022] DELETE /identity/v3/users/5b08841bbfab4a858f735c1ff17df1e4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:51.129775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b113ce-9885-4d04-a716-e7c7672e9907 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:51.130272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b113ce-9885-4d04-a716-e7c7672e9907 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:51.130375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b113ce-9885-4d04-a716-e7c7672e9907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:51.151257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-27b113ce-9885-4d04-a716-e7c7672e9907 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:51.152503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-27b113ce-9885-4d04-a716-e7c7672e9907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:51.211531 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1386/2767] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:51.221377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:51.222788 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:51.279846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe114d97-17f3-4ee5-997c-d3ffa4f94eec admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:51.283653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0035c-74b8-4852-b708-b4d441d80a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:51.284266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0035c-74b8-4852-b708-b4d441d80a44 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:51.284581 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0035c-74b8-4852-b708-b4d441d80a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:51.284855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0035c-74b8-4852-b708-b4d441d80a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:51.294081 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0035c-74b8-4852-b708-b4d441d80a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:51.295604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0035c-74b8-4852-b708-b4d441d80a44 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:51.306543 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1382/2768] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:51.761628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b113ce-9885-4d04-a716-e7c7672e9907 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:51.763072 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-27b113ce-9885-4d04-a716-e7c7672e9907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:51.776986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b113ce-9885-4d04-a716-e7c7672e9907 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:51.782174 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:51.782818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:51.783092 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:51.783359 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] PATH_INFO: `/v3/projects/d5633c9cc15f42eba1d6d4049b5e326b` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:51.784538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5633c9cc15f42eba1d6d4049b5e326b)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:51.790432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:51.898157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5633c9cc15f42eba1d6d4049b5e326b', 'request_id': 'req-6c05ceea-f07d-456e-8006-8e62c23a1e44'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.904437 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5633c9cc15f42eba1d6d4049b5e326b', 'request_id': 'req-6c05ceea-f07d-456e-8006-8e62c23a1e44'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.972028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5633c9cc15f42eba1d6d4049b5e326b', 'request_id': 'req-6c05ceea-f07d-456e-8006-8e62c23a1e44'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.977271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:51.983392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] The token cache is being invalidate because project d5633c9cc15f42eba1d6d4049b5e326b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:35:51.983679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.984372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:51.985855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1387/2769] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:35:51 2022] DELETE /identity/v3/projects/d5633c9cc15f42eba1d6d4049b5e326b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:51.990469 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0035c-74b8-4852-b708-b4d441d80a44 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:52.000555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0035c-74b8-4852-b708-b4d441d80a44 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:52.057316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:52.057985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:52.058297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:52.058576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] PATH_INFO: `/v3/users/b0eddb08ae53408683598c1b6260f784` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:52.059701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0eddb08ae53408683598c1b6260f784)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:52.061781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:52.242080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0eddb08ae53408683598c1b6260f784'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:52.325630 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0eddb08ae53408683598c1b6260f784'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:52.326324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0eddb08ae53408683598c1b6260f784'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:52.326983 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0eddb08ae53408683598c1b6260f784'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:52.334858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d7b22848-c1a5-48a1-8872-a5c879c63590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0eddb08ae53408683598c1b6260f784'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:52.340029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:52.352718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1383/2770] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:35:51 2022] DELETE /identity/v3/users/b0eddb08ae53408683598c1b6260f784 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:52.790900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:52.793209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:52.806883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05ceea-f07d-456e-8006-8e62c23a1e44 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:52.866453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:52.867082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:52.867362 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:52.867664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] PATH_INFO: `/v3/projects/f2057316b9324dc581dd447861c8349e` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:52.868775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2057316b9324dc581dd447861c8349e)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:52.870710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:53.005648 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2057316b9324dc581dd447861c8349e', 'request_id': 'req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:53.012109 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2057316b9324dc581dd447861c8349e', 'request_id': 'req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:53.073310 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2057316b9324dc581dd447861c8349e', 'request_id': 'req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:53.084355 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] The token cache is being invalidate because project f2057316b9324dc581dd447861c8349e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:35:53.084636 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:53.085438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4ed1bd80-19c3-412c-b41a-bfc6f657b94c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:53.086971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1388/2771] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:35:52 2022] DELETE /identity/v3/projects/f2057316b9324dc581dd447861c8349e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:53.092084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe0b6a-c38b-456c-983d-17d9044ade03 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.092529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe0b6a-c38b-456c-983d-17d9044ade03 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.092647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe0b6a-c38b-456c-983d-17d9044ade03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.113431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-7efe0b6a-c38b-456c-983d-17d9044ade03 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:53.114677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-7efe0b6a-c38b-456c-983d-17d9044ade03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:53.175178 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1384/2772] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:53.184212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0151-d6b5-4bac-8ee3-62e46a200140 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.184501 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0151-d6b5-4bac-8ee3-62e46a200140 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.184774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0151-d6b5-4bac-8ee3-62e46a200140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.204273 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-b68d0151-d6b5-4bac-8ee3-62e46a200140 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:53.205485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b68d0151-d6b5-4bac-8ee3-62e46a200140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:53.213386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1389/2773] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:53.217710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe0b6a-c38b-456c-983d-17d9044ade03 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:53.228170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe0b6a-c38b-456c-983d-17d9044ade03 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:53.232778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:53.233378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.233661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.233937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.234617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:35:53.234926 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:53.237161 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:53.248951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1385/2774] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:35:53.256222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d0151-d6b5-4bac-8ee3-62e46a200140 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:53.265447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d0151-d6b5-4bac-8ee3-62e46a200140 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:53.270162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78673a-d2d7-4993-8886-29eecbe3a979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:53.270763 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78673a-d2d7-4993-8886-29eecbe3a979 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.271042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78673a-d2d7-4993-8886-29eecbe3a979 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.271309 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78673a-d2d7-4993-8886-29eecbe3a979 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.272199 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78673a-d2d7-4993-8886-29eecbe3a979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:53.274378 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78673a-d2d7-4993-8886-29eecbe3a979 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:53.419029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1390/2775] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:35:53 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:53.423242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:53.434179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94840f7f-0d15-4197-95ef-bbb5268e821a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:53.496841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c71a28-7834-493b-9fa6-1388d4737d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:53.497454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c71a28-7834-493b-9fa6-1388d4737d58 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.497764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c71a28-7834-493b-9fa6-1388d4737d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.498036 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c71a28-7834-493b-9fa6-1388d4737d58 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.498712 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c71a28-7834-493b-9fa6-1388d4737d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:53.500616 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c71a28-7834-493b-9fa6-1388d4737d58 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:53.588939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1386/2776] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:35:53 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:53.593450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78673a-d2d7-4993-8886-29eecbe3a979 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:53.604599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78673a-d2d7-4993-8886-29eecbe3a979 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:53.609152 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b7d0-0bef-4c8f-b16b-b8ed29acabfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:53.609744 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b7d0-0bef-4c8f-b16b-b8ed29acabfb admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.610019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b7d0-0bef-4c8f-b16b-b8ed29acabfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.610353 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b7d0-0bef-4c8f-b16b-b8ed29acabfb admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.610955 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305b7d0-0bef-4c8f-b16b-b8ed29acabfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:53.612583 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305b7d0-0bef-4c8f-b16b-b8ed29acabfb admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:53.622728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1391/2777] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:53.626872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c71a28-7834-493b-9fa6-1388d4737d58 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:53.637949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c71a28-7834-493b-9fa6-1388d4737d58 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:53.644478 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95ef7a-03c6-4c21-9b4b-5d7d1f8107d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:53.645519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ef7a-03c6-4c21-9b4b-5d7d1f8107d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.646039 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ef7a-03c6-4c21-9b4b-5d7d1f8107d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.646823 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ef7a-03c6-4c21-9b4b-5d7d1f8107d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.648014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95ef7a-03c6-4c21-9b4b-5d7d1f8107d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:53.649996 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95ef7a-03c6-4c21-9b4b-5d7d1f8107d4 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:53.664500 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1387/2778] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:53.668954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b7d0-0bef-4c8f-b16b-b8ed29acabfb admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:53.681125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b7d0-0bef-4c8f-b16b-b8ed29acabfb admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:53.685894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9db772-ab9f-4737-a24b-871dfc7767d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:53.686611 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9db772-ab9f-4737-a24b-871dfc7767d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.686948 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9db772-ab9f-4737-a24b-871dfc7767d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.687266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9db772-ab9f-4737-a24b-871dfc7767d8 admin admin] PATH_INFO: `/v3/projects/1c0331c156784ef0b09ac499c9875e29/users/f32125c3fa4a4965a50b2dcdebc9e406/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.702584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9db772-ab9f-4737-a24b-871dfc7767d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c0331c156784ef0b09ac499c9875e29, user_id=f32125c3fa4a4965a50b2dcdebc9e406, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:53.705823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9db772-ab9f-4737-a24b-871dfc7767d8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:53.778038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1392/2779] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:53 2022] PUT /identity/v3/projects/1c0331c156784ef0b09ac499c9875e29/users/f32125c3fa4a4965a50b2dcdebc9e406/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:53.788373 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a81348-9e5c-4909-bcc8-b9ec30601aa0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.789257 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a81348-9e5c-4909-bcc8-b9ec30601aa0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.789519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a81348-9e5c-4909-bcc8-b9ec30601aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.816971 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-d1a81348-9e5c-4909-bcc8-b9ec30601aa0 None None] MFA Rules not processed for user `f32125c3fa4a4965a50b2dcdebc9e406`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:53.818221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d1a81348-9e5c-4909-bcc8-b9ec30601aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:53.861308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:53.885447 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1388/2780] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:53.889543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9db772-ab9f-4737-a24b-871dfc7767d8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:53.915882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:53.952522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9db772-ab9f-4737-a24b-871dfc7767d8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:53.957841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d89b2-2939-4cd3-b2c5-c00b8f480d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:53.958523 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d89b2-2939-4cd3-b2c5-c00b8f480d12 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:53.965198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d89b2-2939-4cd3-b2c5-c00b8f480d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:53.965198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d89b2-2939-4cd3-b2c5-c00b8f480d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:53.965198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d89b2-2939-4cd3-b2c5-c00b8f480d12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:53.965198 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d89b2-2939-4cd3-b2c5-c00b8f480d12 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.082678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1393/2781] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:35:53 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:54.087150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a81348-9e5c-4909-bcc8-b9ec30601aa0 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.098232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a81348-9e5c-4909-bcc8-b9ec30601aa0 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:54.154572 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2914e-e1b1-4a18-843e-62301ca30c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:54.155180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2914e-e1b1-4a18-843e-62301ca30c1d admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.155456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2914e-e1b1-4a18-843e-62301ca30c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.155727 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2914e-e1b1-4a18-843e-62301ca30c1d admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.156353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2914e-e1b1-4a18-843e-62301ca30c1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:54.158419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2914e-e1b1-4a18-843e-62301ca30c1d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.234228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1389/2782] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:35:54 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:54.242742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d89b2-2939-4cd3-b2c5-c00b8f480d12 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.253869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d89b2-2939-4cd3-b2c5-c00b8f480d12 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:54.258606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa41ee-bb7e-414b-bfab-49c923af11c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:54.259207 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa41ee-bb7e-414b-bfab-49c923af11c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.259489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa41ee-bb7e-414b-bfab-49c923af11c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.259792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa41ee-bb7e-414b-bfab-49c923af11c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.263187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa41ee-bb7e-414b-bfab-49c923af11c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:54.263187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa41ee-bb7e-414b-bfab-49c923af11c5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.273065 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1394/2783] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:54.277104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2914e-e1b1-4a18-843e-62301ca30c1d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.287349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2914e-e1b1-4a18-843e-62301ca30c1d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:54.291963 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa06e8-d027-438c-b77c-785023b32ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:54.292559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa06e8-d027-438c-b77c-785023b32ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.292840 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa06e8-d027-438c-b77c-785023b32ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.293113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa06e8-d027-438c-b77c-785023b32ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.293752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa06e8-d027-438c-b77c-785023b32ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:54.295364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa06e8-d027-438c-b77c-785023b32ea7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.306252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1390/2784] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:54.310366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa41ee-bb7e-414b-bfab-49c923af11c5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.320987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa41ee-bb7e-414b-bfab-49c923af11c5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:54.325630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab957-ed1e-4d83-86e7-b564bfcac03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:54.326350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab957-ed1e-4d83-86e7-b564bfcac03b admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.326634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab957-ed1e-4d83-86e7-b564bfcac03b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.326923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab957-ed1e-4d83-86e7-b564bfcac03b admin admin] PATH_INFO: `/v3/projects/7d0d46eb0767428dbbf0c8d4f6ea6a2a/users/e1364d066df241d689ada93c22975a6c/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.342394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ab957-ed1e-4d83-86e7-b564bfcac03b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d0d46eb0767428dbbf0c8d4f6ea6a2a, user_id=e1364d066df241d689ada93c22975a6c, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:54.345021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ab957-ed1e-4d83-86e7-b564bfcac03b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.410671 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1395/2785] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:54 2022] PUT /identity/v3/projects/7d0d46eb0767428dbbf0c8d4f6ea6a2a/users/e1364d066df241d689ada93c22975a6c/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:54.422047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f5a48-1778-4f80-b7aa-ec793ffe2eff None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.422366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f5a48-1778-4f80-b7aa-ec793ffe2eff None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.422647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f5a48-1778-4f80-b7aa-ec793ffe2eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.451163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-333f5a48-1778-4f80-b7aa-ec793ffe2eff None None] MFA Rules not processed for user `e1364d066df241d689ada93c22975a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:54.452382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-333f5a48-1778-4f80-b7aa-ec793ffe2eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:54.517612 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1391/2786] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:54.522915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab957-ed1e-4d83-86e7-b564bfcac03b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.585598 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab957-ed1e-4d83-86e7-b564bfcac03b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:54.590368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52da00-69be-4f95-8c4d-ad5bf2841035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:54.591008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52da00-69be-4f95-8c4d-ad5bf2841035 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.591287 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52da00-69be-4f95-8c4d-ad5bf2841035 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.591557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52da00-69be-4f95-8c4d-ad5bf2841035 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.592186 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52da00-69be-4f95-8c4d-ad5bf2841035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:54.594123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52da00-69be-4f95-8c4d-ad5bf2841035 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.676442 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1396/2787] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:35:54 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:54.680564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f5a48-1778-4f80-b7aa-ec793ffe2eff None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.691466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f5a48-1778-4f80-b7aa-ec793ffe2eff None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:54.738678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:54.748527 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7285e-32e4-4310-b85e-7ed6f803ca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:54.749079 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7285e-32e4-4310-b85e-7ed6f803ca3b admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.749362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7285e-32e4-4310-b85e-7ed6f803ca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.749636 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7285e-32e4-4310-b85e-7ed6f803ca3b admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.750311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7285e-32e4-4310-b85e-7ed6f803ca3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:54.752269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7285e-32e4-4310-b85e-7ed6f803ca3b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.834934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1392/2788] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:35:54 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:35:54.839071 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52da00-69be-4f95-8c4d-ad5bf2841035 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.850485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52da00-69be-4f95-8c4d-ad5bf2841035 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:54.855223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb3e5c-44ff-43f9-83fe-f6aaa713e0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:54.855877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb3e5c-44ff-43f9-83fe-f6aaa713e0ec admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.856165 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb3e5c-44ff-43f9-83fe-f6aaa713e0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.856459 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb3e5c-44ff-43f9-83fe-f6aaa713e0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.857191 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb3e5c-44ff-43f9-83fe-f6aaa713e0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:54.858797 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb3e5c-44ff-43f9-83fe-f6aaa713e0ec admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.871069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1397/2789] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:54.875426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7285e-32e4-4310-b85e-7ed6f803ca3b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.885517 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7285e-32e4-4310-b85e-7ed6f803ca3b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:54.890364 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1042b9-4e5b-4b19-acc6-71d6362c54c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:54.891009 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1042b9-4e5b-4b19-acc6-71d6362c54c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:54.891285 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1042b9-4e5b-4b19-acc6-71d6362c54c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:54.891558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1042b9-4e5b-4b19-acc6-71d6362c54c9 admin admin] PATH_INFO: `/v3/projects/6b932fd0a7ca43afba18e78f4313b04d/users/324a8e065dd64b59a1b468a73d244ceb/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:54.906471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1042b9-4e5b-4b19-acc6-71d6362c54c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b932fd0a7ca43afba18e78f4313b04d, user_id=324a8e065dd64b59a1b468a73d244ceb, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:54.909626 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1042b9-4e5b-4b19-acc6-71d6362c54c9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:54.971889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1393/2790] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:54 2022] PUT /identity/v3/projects/6b932fd0a7ca43afba18e78f4313b04d/users/324a8e065dd64b59a1b468a73d244ceb/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:54.975843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb3e5c-44ff-43f9-83fe-f6aaa713e0ec admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:54.985802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb3e5c-44ff-43f9-83fe-f6aaa713e0ec admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:55.035275 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:55.050584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b2af6-3097-4a1c-8618-56d5acc2a5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:55.051194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b2af6-3097-4a1c-8618-56d5acc2a5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:55.051472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b2af6-3097-4a1c-8618-56d5acc2a5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:55.051745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b2af6-3097-4a1c-8618-56d5acc2a5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:55.052394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b2af6-3097-4a1c-8618-56d5acc2a5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:55.053993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b2af6-3097-4a1c-8618-56d5acc2a5d9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:55.064167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1398/2791] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:35:55.068407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1042b9-4e5b-4b19-acc6-71d6362c54c9 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:55.078638 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1042b9-4e5b-4b19-acc6-71d6362c54c9 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:55.083969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:55.084609 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:55.084889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:55.085162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] PATH_INFO: `/v3/projects/6b932fd0a7ca43afba18e78f4313b04d/users/324a8e065dd64b59a1b468a73d244ceb/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:55.087230 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b932fd0a7ca43afba18e78f4313b04d, user_id=324a8e065dd64b59a1b468a73d244ceb, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:55.089850 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:55.100728 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1394/2792] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:35:55 2022] PUT /identity/v3/projects/6b932fd0a7ca43afba18e78f4313b04d/users/324a8e065dd64b59a1b468a73d244ceb/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:35:55.111821 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65158a-514c-4afc-a94a-2ebf2240ca0b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:55.112451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65158a-514c-4afc-a94a-2ebf2240ca0b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:55.112451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65158a-514c-4afc-a94a-2ebf2240ca0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:55.140585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-4f65158a-514c-4afc-a94a-2ebf2240ca0b None None] MFA Rules not processed for user `324a8e065dd64b59a1b468a73d244ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:35:55.141825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4f65158a-514c-4afc-a94a-2ebf2240ca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:55.210705 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1399/2793] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:35:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:35:55.222413 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:55.224105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:55.282344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d78fb-5c31-4807-8fc3-aaa8c12e79d2 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:55.286703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff29e7-7cff-44dc-85dc-53cbc77df561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:55.287308 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff29e7-7cff-44dc-85dc-53cbc77df561 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:55.287590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff29e7-7cff-44dc-85dc-53cbc77df561 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:55.287862 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff29e7-7cff-44dc-85dc-53cbc77df561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:55.297059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff29e7-7cff-44dc-85dc-53cbc77df561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:55.298520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff29e7-7cff-44dc-85dc-53cbc77df561 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:55.308117 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1395/2794] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:55.454717 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65158a-514c-4afc-a94a-2ebf2240ca0b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:55.456718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4f65158a-514c-4afc-a94a-2ebf2240ca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:35:55.468626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65158a-514c-4afc-a94a-2ebf2240ca0b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:55.472274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324940ca-882b-4361-beb1-f75896591f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:55.472874 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324940ca-882b-4361-beb1-f75896591f48 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:55.473156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324940ca-882b-4361-beb1-f75896591f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:55.473429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324940ca-882b-4361-beb1-f75896591f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:55.529851 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324940ca-882b-4361-beb1-f75896591f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:55.531302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324940ca-882b-4361-beb1-f75896591f48 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:55.540123 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1400/2795] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:35:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:35:59.586256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff29e7-7cff-44dc-85dc-53cbc77df561 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:35:59.651925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff29e7-7cff-44dc-85dc-53cbc77df561 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:35:59.656497 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:35:59.657152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:35:59.657401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:35:59.657674 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] PATH_INFO: `/v3/users/f32125c3fa4a4965a50b2dcdebc9e406` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:35:59.659463 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32125c3fa4a4965a50b2dcdebc9e406)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:35:59.661383 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:35:59.865658 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32125c3fa4a4965a50b2dcdebc9e406'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:59.867564 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:59.977085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32125c3fa4a4965a50b2dcdebc9e406'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:59.977706 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32125c3fa4a4965a50b2dcdebc9e406'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:59.978352 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32125c3fa4a4965a50b2dcdebc9e406'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:35:59.986380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32125c3fa4a4965a50b2dcdebc9e406'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:00.004865 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1396/2796] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:35:59 2022] DELETE /identity/v3/users/f32125c3fa4a4965a50b2dcdebc9e406 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:00.009430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33cf66-1607-4f46-ac78-98ebbe43bbef None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:00.009929 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33cf66-1607-4f46-ac78-98ebbe43bbef None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:00.010019 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33cf66-1607-4f46-ac78-98ebbe43bbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:00.029748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-aa33cf66-1607-4f46-ac78-98ebbe43bbef None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:00.030900 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-aa33cf66-1607-4f46-ac78-98ebbe43bbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:00.090194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1401/2797] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:00.100062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:00.101578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:00.160471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb73bd-17c9-4681-a4b2-91655443d171 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:00.164344 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6216fb-23a6-4cac-88f9-0d030c77ad18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:00.164952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6216fb-23a6-4cac-88f9-0d030c77ad18 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:00.165237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6216fb-23a6-4cac-88f9-0d030c77ad18 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:00.165512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6216fb-23a6-4cac-88f9-0d030c77ad18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:00.174647 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6216fb-23a6-4cac-88f9-0d030c77ad18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:00.176087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6216fb-23a6-4cac-88f9-0d030c77ad18 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:00.185477 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1397/2798] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:36:00.630373 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33cf66-1607-4f46-ac78-98ebbe43bbef None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:00.632637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-aa33cf66-1607-4f46-ac78-98ebbe43bbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:00.647902 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33cf66-1607-4f46-ac78-98ebbe43bbef None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:00.653722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:00.654534 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:00.654743 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:00.655023 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] PATH_INFO: `/v3/projects/1c0331c156784ef0b09ac499c9875e29` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:00.656389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c0331c156784ef0b09ac499c9875e29)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:00.658324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:00.660704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:00.782861 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c0331c156784ef0b09ac499c9875e29', 'request_id': 'req-c603181e-04c0-4392-912a-7b65b62f07a7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:00.790025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c0331c156784ef0b09ac499c9875e29', 'request_id': 'req-c603181e-04c0-4392-912a-7b65b62f07a7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:00.857125 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c0331c156784ef0b09ac499c9875e29', 'request_id': 'req-c603181e-04c0-4392-912a-7b65b62f07a7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:00.869909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] The token cache is being invalidate because project 1c0331c156784ef0b09ac499c9875e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:36:00.870252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:00.870923 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:00.872654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1402/2799] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:36:00 2022] DELETE /identity/v3/projects/1c0331c156784ef0b09ac499c9875e29 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:00.876597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6216fb-23a6-4cac-88f9-0d030c77ad18 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:00.887791 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6216fb-23a6-4cac-88f9-0d030c77ad18 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:00.945754 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:00.946699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:00.946699 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:00.946991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] PATH_INFO: `/v3/users/e1364d066df241d689ada93c22975a6c` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:00.948141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1364d066df241d689ada93c22975a6c)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:00.950073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:01.116901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1364d066df241d689ada93c22975a6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.188384 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1364d066df241d689ada93c22975a6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.188937 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1364d066df241d689ada93c22975a6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.189540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1364d066df241d689ada93c22975a6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.197167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1364d066df241d689ada93c22975a6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.202449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:01.214990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1398/2800] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:36:00 2022] DELETE /identity/v3/users/e1364d066df241d689ada93c22975a6c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:01.640976 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:01.642558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:01.656554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603181e-04c0-4392-912a-7b65b62f07a7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:01.714046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:01.714627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:01.714905 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:01.715179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] PATH_INFO: `/v3/projects/7d0d46eb0767428dbbf0c8d4f6ea6a2a` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:01.716282 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d0d46eb0767428dbbf0c8d4f6ea6a2a)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:01.718187 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:01.845428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d0d46eb0767428dbbf0c8d4f6ea6a2a', 'request_id': 'req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.851689 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d0d46eb0767428dbbf0c8d4f6ea6a2a', 'request_id': 'req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.960029 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d0d46eb0767428dbbf0c8d4f6ea6a2a', 'request_id': 'req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.976885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] The token cache is being invalidate because project 7d0d46eb0767428dbbf0c8d4f6ea6a2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:36:01.976885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.976885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:01.976885 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1403/2801] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:36:01 2022] DELETE /identity/v3/projects/7d0d46eb0767428dbbf0c8d4f6ea6a2a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:01.979803 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:01.981199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:01.994942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31c0e2-22d8-459d-ae10-a4c46e6ce53b admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:02.051751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:02.052388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:02.052675 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:02.052952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] PATH_INFO: `/v3/users/324a8e065dd64b59a1b468a73d244ceb` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:02.054098 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324a8e065dd64b59a1b468a73d244ceb)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:02.056204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:02.248639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324a8e065dd64b59a1b468a73d244ceb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:02.349139 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324a8e065dd64b59a1b468a73d244ceb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:02.349751 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324a8e065dd64b59a1b468a73d244ceb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:02.350402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324a8e065dd64b59a1b468a73d244ceb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:02.358063 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0bf1c4e-4766-45b1-a53c-411f529db420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324a8e065dd64b59a1b468a73d244ceb'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:02.376853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1399/2802] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:36:01 2022] DELETE /identity/v3/users/324a8e065dd64b59a1b468a73d244ceb => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:02.784871 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:02.787276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:02.802601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7892a-41c5-41d1-90f1-4f446c8cb74c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:02.823005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:02.862272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:02.862913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:02.863216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:02.863475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] PATH_INFO: `/v3/projects/6b932fd0a7ca43afba18e78f4313b04d` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:02.864628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b932fd0a7ca43afba18e78f4313b04d)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:02.866581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:02.984272 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b932fd0a7ca43afba18e78f4313b04d', 'request_id': 'req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:02.992116 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b932fd0a7ca43afba18e78f4313b04d', 'request_id': 'req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:03.062935 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b932fd0a7ca43afba18e78f4313b04d', 'request_id': 'req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:03.076393 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] The token cache is being invalidate because project 6b932fd0a7ca43afba18e78f4313b04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:36:03.076681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:03.077402 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-43b1b6a6-fcbe-4c36-8282-f6c03d3aacd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:03.079042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1404/2803] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:36:02 2022] DELETE /identity/v3/projects/6b932fd0a7ca43afba18e78f4313b04d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:03.085623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65495e-463e-4e49-9bec-fbe5809a6230 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.086125 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65495e-463e-4e49-9bec-fbe5809a6230 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.086250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65495e-463e-4e49-9bec-fbe5809a6230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.105723 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0e65495e-463e-4e49-9bec-fbe5809a6230 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:03.107164 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0e65495e-463e-4e49-9bec-fbe5809a6230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:03.166559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1400/2804] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:03.177934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc3916-d41a-48fc-8bf2-50a1cf1eae63 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.178252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc3916-d41a-48fc-8bf2-50a1cf1eae63 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.178553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc3916-d41a-48fc-8bf2-50a1cf1eae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.199226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-edcc3916-d41a-48fc-8bf2-50a1cf1eae63 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:03.200419 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-edcc3916-d41a-48fc-8bf2-50a1cf1eae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:03.208464 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1405/2805] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:03.213244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65495e-463e-4e49-9bec-fbe5809a6230 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:03.223793 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65495e-463e-4e49-9bec-fbe5809a6230 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:03.228487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:03.230863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.230863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.230863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.230863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:36:03.230863 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:03.232212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:03.233778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:03.245495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1401/2806] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:36:03.250226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc3916-d41a-48fc-8bf2-50a1cf1eae63 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:03.260183 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc3916-d41a-48fc-8bf2-50a1cf1eae63 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:03.264800 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b526aba-27ec-4bf4-b7df-9c610caf9941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:03.265383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b526aba-27ec-4bf4-b7df-9c610caf9941 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.265656 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b526aba-27ec-4bf4-b7df-9c610caf9941 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.265921 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b526aba-27ec-4bf4-b7df-9c610caf9941 admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.266584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b526aba-27ec-4bf4-b7df-9c610caf9941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:03.268505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b526aba-27ec-4bf4-b7df-9c610caf9941 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:03.390278 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1406/2807] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:36:03 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:36:03.394518 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:03.406435 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396e3b0-5f3f-4251-a9c5-5e7dfbedc965 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:03.464656 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a8334-b0ef-454c-b5c1-71fa56c2039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:03.466187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a8334-b0ef-454c-b5c1-71fa56c2039e admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.466187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a8334-b0ef-454c-b5c1-71fa56c2039e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.466187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a8334-b0ef-454c-b5c1-71fa56c2039e admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.466524 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a8334-b0ef-454c-b5c1-71fa56c2039e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:03.468445 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a8334-b0ef-454c-b5c1-71fa56c2039e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:03.589540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1402/2808] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:36:03 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:36:03.593726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b526aba-27ec-4bf4-b7df-9c610caf9941 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:03.604513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b526aba-27ec-4bf4-b7df-9c610caf9941 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:03.609136 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94945931-c863-4413-97cd-85fddb824671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:03.610257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94945931-c863-4413-97cd-85fddb824671 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.610257 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94945931-c863-4413-97cd-85fddb824671 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.610574 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94945931-c863-4413-97cd-85fddb824671 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.610934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94945931-c863-4413-97cd-85fddb824671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:03.612522 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94945931-c863-4413-97cd-85fddb824671 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:03.623499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1407/2809] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:36:03.627618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a8334-b0ef-454c-b5c1-71fa56c2039e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:03.638010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a8334-b0ef-454c-b5c1-71fa56c2039e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:03.642851 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a09ea-43df-4e70-b64f-db0877e0b0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:03.643401 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a09ea-43df-4e70-b64f-db0877e0b0cc admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.643685 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a09ea-43df-4e70-b64f-db0877e0b0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.643968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a09ea-43df-4e70-b64f-db0877e0b0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.644668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a09ea-43df-4e70-b64f-db0877e0b0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:03.646311 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a09ea-43df-4e70-b64f-db0877e0b0cc admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:03.657732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1403/2810] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:36:03.662544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94945931-c863-4413-97cd-85fddb824671 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:03.672375 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94945931-c863-4413-97cd-85fddb824671 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:03.677465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:03.678097 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.678403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.678674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] PATH_INFO: `/v3/projects/c7b029c3495b4a3ba5172535d31fa731/users/3af1f48991c741d49632f99292d56b03/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.692962 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b029c3495b4a3ba5172535d31fa731, user_id=3af1f48991c741d49632f99292d56b03, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:03.695677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:03.766630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1408/2811] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:36:03 2022] PUT /identity/v3/projects/c7b029c3495b4a3ba5172535d31fa731/users/3af1f48991c741d49632f99292d56b03/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:03.780045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50272134-f745-4ff2-9997-6ba3d15c04e7 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.780334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50272134-f745-4ff2-9997-6ba3d15c04e7 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.780610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50272134-f745-4ff2-9997-6ba3d15c04e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.808147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-50272134-f745-4ff2-9997-6ba3d15c04e7 None None] MFA Rules not processed for user `3af1f48991c741d49632f99292d56b03`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:03.809353 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-50272134-f745-4ff2-9997-6ba3d15c04e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:03.872718 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1404/2812] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:03.885104 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:03.886893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:03.939429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:03.945233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35879f40-bf76-48ac-b273-dc2cfda20f1e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:03.948942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfdf35-2df3-4dbb-9f1b-92c69f44a0c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c07c85b63514d10bb17b836c60f2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:03.949555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfdf35-2df3-4dbb-9f1b-92c69f44a0c7 service glance] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:03.949840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfdf35-2df3-4dbb-9f1b-92c69f44a0c7 service glance] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:03.950115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfdf35-2df3-4dbb-9f1b-92c69f44a0c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:03.958559 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfdf35-2df3-4dbb-9f1b-92c69f44a0c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:03.960010 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfdf35-2df3-4dbb-9f1b-92c69f44a0c7 service glance] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:03.969664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1409/2813] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:36:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:36:13.869919 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:14.011319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:25.368970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50272134-f745-4ff2-9997-6ba3d15c04e7 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:25.429789 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50272134-f745-4ff2-9997-6ba3d15c04e7 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:25.434235 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:25.434858 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:25.435136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:25.435406 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] PATH_INFO: `/v3/users/3af1f48991c741d49632f99292d56b03` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:25.436496 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af1f48991c741d49632f99292d56b03)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:25.438428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:25.620151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af1f48991c741d49632f99292d56b03'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:25.674317 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af1f48991c741d49632f99292d56b03'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:25.674975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af1f48991c741d49632f99292d56b03'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:25.676199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af1f48991c741d49632f99292d56b03'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:25.683632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af1f48991c741d49632f99292d56b03'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:25.701250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1405/2814] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:36:25 2022] DELETE /identity/v3/users/3af1f48991c741d49632f99292d56b03 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:25.705791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9163-54ba-4e7e-aeaa-0438b7d57a13 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:25.706294 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9163-54ba-4e7e-aeaa-0438b7d57a13 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:25.706383 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc9163-54ba-4e7e-aeaa-0438b7d57a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:25.726204 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-7ddc9163-54ba-4e7e-aeaa-0438b7d57a13 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:25.727403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7ddc9163-54ba-4e7e-aeaa-0438b7d57a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:25.786202 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1410/2815] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:25.796142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:25.797551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:25.863291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee37a68-66b2-4c8a-8dcd-852d991506f2 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:25.866984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d306a17-2d4b-4a18-8050-346489b4c4ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:25.867587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306a17-2d4b-4a18-8050-346489b4c4ff service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:25.867869 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306a17-2d4b-4a18-8050-346489b4c4ff service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:25.868144 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306a17-2d4b-4a18-8050-346489b4c4ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:25.877128 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d306a17-2d4b-4a18-8050-346489b4c4ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:25.878535 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d306a17-2d4b-4a18-8050-346489b4c4ff service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:25.887978 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1406/2816] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:36:26.293414 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc9163-54ba-4e7e-aeaa-0438b7d57a13 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:26.294901 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-7ddc9163-54ba-4e7e-aeaa-0438b7d57a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:26.307211 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc9163-54ba-4e7e-aeaa-0438b7d57a13 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:26.312664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:26.313380 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:26.313736 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:26.314087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] PATH_INFO: `/v3/projects/c7b029c3495b4a3ba5172535d31fa731` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:26.315372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b029c3495b4a3ba5172535d31fa731)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:26.318539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:26.412691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b029c3495b4a3ba5172535d31fa731', 'request_id': 'req-258de7a3-d9df-4345-b170-512f4dcb27f3'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:26.418731 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b029c3495b4a3ba5172535d31fa731', 'request_id': 'req-258de7a3-d9df-4345-b170-512f4dcb27f3'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:26.468750 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b029c3495b4a3ba5172535d31fa731', 'request_id': 'req-258de7a3-d9df-4345-b170-512f4dcb27f3'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:26.480873 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] The token cache is being invalidate because project c7b029c3495b4a3ba5172535d31fa731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:36:26.481941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:26.481941 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-258de7a3-d9df-4345-b170-512f4dcb27f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:26.483476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1411/2817] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:36:26 2022] DELETE /identity/v3/projects/c7b029c3495b4a3ba5172535d31fa731 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:26.488412 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfe40-112f-480a-98e3-bf262bbd07e2 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:26.489466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfe40-112f-480a-98e3-bf262bbd07e2 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:26.489466 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfe40-112f-480a-98e3-bf262bbd07e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:26.511825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-25cdfe40-112f-480a-98e3-bf262bbd07e2 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:26.513066 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-25cdfe40-112f-480a-98e3-bf262bbd07e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:26.521986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:26.577559 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1407/2818] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:26.585710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f6c37-127a-4d63-ada7-1104a5020666 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:26.586021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f6c37-127a-4d63-ada7-1104a5020666 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:26.586484 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f6c37-127a-4d63-ada7-1104a5020666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:26.606832 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-bf9f6c37-127a-4d63-ada7-1104a5020666 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:26.608038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bf9f6c37-127a-4d63-ada7-1104a5020666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:26.616546 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1412/2819] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:26.620804 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdfe40-112f-480a-98e3-bf262bbd07e2 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:26.631923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdfe40-112f-480a-98e3-bf262bbd07e2 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:26.636342 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:26.638270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:26.638270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:26.638270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:26.638270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:36:26.638764 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:26.645885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:26.653688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1408/2820] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:36:26.658100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f6c37-127a-4d63-ada7-1104a5020666 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:26.667498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f6c37-127a-4d63-ada7-1104a5020666 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:26.672075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3401c5-c72d-4947-ba9c-a95372eef66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:26.672668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3401c5-c72d-4947-ba9c-a95372eef66a admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:26.673035 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3401c5-c72d-4947-ba9c-a95372eef66a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:26.673312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3401c5-c72d-4947-ba9c-a95372eef66a admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:26.673942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3401c5-c72d-4947-ba9c-a95372eef66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:26.675882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3401c5-c72d-4947-ba9c-a95372eef66a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:26.784420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1413/2821] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:36:26 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:36:26.788709 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:26.799758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b2ff6-5797-4662-8330-cf7bd98858d6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:26.856510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e57cc-f74d-4a31-bf50-17409d8c7079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:26.857119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e57cc-f74d-4a31-bf50-17409d8c7079 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:26.857400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e57cc-f74d-4a31-bf50-17409d8c7079 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:26.857678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e57cc-f74d-4a31-bf50-17409d8c7079 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:26.858347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e57cc-f74d-4a31-bf50-17409d8c7079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:26.860264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e57cc-f74d-4a31-bf50-17409d8c7079 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:26.966187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1409/2822] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:36:26 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:36:26.970239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3401c5-c72d-4947-ba9c-a95372eef66a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:26.980621 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3401c5-c72d-4947-ba9c-a95372eef66a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:26.985267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb8841-d22a-4f53-bdc4-de1f6e7e2db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:26.985864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb8841-d22a-4f53-bdc4-de1f6e7e2db2 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:26.986164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb8841-d22a-4f53-bdc4-de1f6e7e2db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:26.986444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb8841-d22a-4f53-bdc4-de1f6e7e2db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:26.987085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb8841-d22a-4f53-bdc4-de1f6e7e2db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:26.988668 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb8841-d22a-4f53-bdc4-de1f6e7e2db2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:26.998429 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1414/2823] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:36:27.002305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e57cc-f74d-4a31-bf50-17409d8c7079 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:27.013149 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e57cc-f74d-4a31-bf50-17409d8c7079 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:27.017876 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd22ec-4e4a-4cdd-9571-3036178e55eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:27.018534 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd22ec-4e4a-4cdd-9571-3036178e55eb admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:27.018819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd22ec-4e4a-4cdd-9571-3036178e55eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:27.019089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd22ec-4e4a-4cdd-9571-3036178e55eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:27.019747 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd22ec-4e4a-4cdd-9571-3036178e55eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:27.021395 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd22ec-4e4a-4cdd-9571-3036178e55eb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:27.033220 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1410/2824] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:36:27.038589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb8841-d22a-4f53-bdc4-de1f6e7e2db2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:27.049290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb8841-d22a-4f53-bdc4-de1f6e7e2db2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:27.053969 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:27.054664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:27.054938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:27.055209 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] PATH_INFO: `/v3/projects/c690384cd06642628d5764761dc50d67/users/a4649b5cfb454ea1a2fadad4cda17995/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:27.070111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c690384cd06642628d5764761dc50d67, user_id=a4649b5cfb454ea1a2fadad4cda17995, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:27.072786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:27.172021 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1415/2825] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:36:27 2022] PUT /identity/v3/projects/c690384cd06642628d5764761dc50d67/users/a4649b5cfb454ea1a2fadad4cda17995/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:36:27.183140 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4635d-fed5-4153-af37-3cac3404b2b9 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:27.183666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4635d-fed5-4153-af37-3cac3404b2b9 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:27.183948 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4635d-fed5-4153-af37-3cac3404b2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:27.211657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-3fa4635d-fed5-4153-af37-3cac3404b2b9 None None] MFA Rules not processed for user `a4649b5cfb454ea1a2fadad4cda17995`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:27.212877 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-3fa4635d-fed5-4153-af37-3cac3404b2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:27.277176 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1411/2826] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:36:27.288663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:27.290894 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:27.314391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:27.350805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b380c-3a41-48c4-ae83-9afe272ba9a8 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:27.357166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:27.357166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:27.357166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:27.357166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:27.366723 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:27.368171 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:27.377882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1416/2827] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:36:29.048618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d723-f0e8-4092-865e-65fda6bba220 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:29.049013 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d723-f0e8-4092-865e-65fda6bba220 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:29.049179 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752d723-f0e8-4092-865e-65fda6bba220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:29.068914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-0752d723-f0e8-4092-865e-65fda6bba220 None None] MFA Rules not processed for user `a4649b5cfb454ea1a2fadad4cda17995`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:29.070204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0752d723-f0e8-4092-865e-65fda6bba220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:29.077051 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1412/2828] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:30.905517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:30.907052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:30.988423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be266cee-5f67-469c-93e9-87469ff246d6 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:30.993444 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:30.994051 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:30.994364 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:30.994643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:30.996369 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:31.008041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:31.009628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:31.019920 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1417/2829] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:36:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:36:37.467843 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752d723-f0e8-4092-865e-65fda6bba220 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:37.469377 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-0752d723-f0e8-4092-865e-65fda6bba220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:37.474472 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:37.527569 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752d723-f0e8-4092-865e-65fda6bba220 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:37.531319 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f409336-104b-4fce-8d9b-56607a048837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:37.531910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409336-104b-4fce-8d9b-56607a048837 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:37.532186 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409336-104b-4fce-8d9b-56607a048837 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:37.532454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409336-104b-4fce-8d9b-56607a048837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:37.534199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-6f409336-104b-4fce-8d9b-56607a048837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:37.543968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f409336-104b-4fce-8d9b-56607a048837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:37.545375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f409336-104b-4fce-8d9b-56607a048837 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:37.553951 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1413/2830] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:36:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:36:41.015460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=49 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:44.267185 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:44.268818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:44.343034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5785a9-76b3-4c90-98d0-ef3bfa57d168 service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:44.348540 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:44.349147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:44.349424 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:44.349691 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:44.351674 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:44.367862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:44.369855 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:44.385315 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1418/2831] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:36:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:36:47.582331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=25 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:51.365623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c4bae-e780-4590-b94e-c0e0464a7971 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:51.366248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c4bae-e780-4590-b94e-c0e0464a7971 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:51.366248 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c4bae-e780-4590-b94e-c0e0464a7971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:51.386932 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-442c4bae-e780-4590-b94e-c0e0464a7971 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:51.388335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-442c4bae-e780-4590-b94e-c0e0464a7971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:51.444822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1414/2832] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:51.567929 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:51.569605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:51.640451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c2fdc-dac6-4b52-9fed-b13109d49748 service nova] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:51.645550 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0da5d-6f83-45a5-b3aa-098d2ec03df6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:51.646192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0da5d-6f83-45a5-b3aa-098d2ec03df6 service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:51.646475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0da5d-6f83-45a5-b3aa-098d2ec03df6 service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:51.646749 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0da5d-6f83-45a5-b3aa-098d2ec03df6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:51.655992 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0da5d-6f83-45a5-b3aa-098d2ec03df6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:51.657588 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0da5d-6f83-45a5-b3aa-098d2ec03df6 service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:51.665877 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1419/2833] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:36:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:36:52.508721 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30daa80-502d-42ad-8504-14fb802e2326 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:52.509289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30daa80-502d-42ad-8504-14fb802e2326 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:52.509289 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30daa80-502d-42ad-8504-14fb802e2326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:52.529284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f30daa80-502d-42ad-8504-14fb802e2326 None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:52.530537 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f30daa80-502d-42ad-8504-14fb802e2326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:52.537820 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1415/2834] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 14:36:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:52.550614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0da5d-6f83-45a5-b3aa-098d2ec03df6 service swift] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:52.561248 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0da5d-6f83-45a5-b3aa-098d2ec03df6 service swift] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:52.566433 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d91f2-47ce-4ad4-ada4-81b4cff2cbb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:52.567028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d91f2-47ce-4ad4-ada4-81b4cff2cbb9 service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:52.567325 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d91f2-47ce-4ad4-ada4-81b4cff2cbb9 service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:52.567580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d91f2-47ce-4ad4-ada4-81b4cff2cbb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:52.575628 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d91f2-47ce-4ad4-ada4-81b4cff2cbb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:52.577201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d91f2-47ce-4ad4-ada4-81b4cff2cbb9 service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:52.585256 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1420/2835] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:36:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:36:53.309367 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33743-0485-4bf0-b40c-f8f73bd4506c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:53.309684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33743-0485-4bf0-b40c-f8f73bd4506c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:53.309962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33743-0485-4bf0-b40c-f8f73bd4506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:53.330103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-04d33743-0485-4bf0-b40c-f8f73bd4506c None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:53.331449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-04d33743-0485-4bf0-b40c-f8f73bd4506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:53.338905 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1416/2836] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 14:36:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:53.350841 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d91f2-47ce-4ad4-ada4-81b4cff2cbb9 service swift] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:53.361995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d91f2-47ce-4ad4-ada4-81b4cff2cbb9 service swift] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:53.367066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e756df-f065-4fe2-ac95-e5a61ffa0d57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:53.367654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e756df-f065-4fe2-ac95-e5a61ffa0d57 service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:53.367932 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e756df-f065-4fe2-ac95-e5a61ffa0d57 service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:53.368233 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e756df-f065-4fe2-ac95-e5a61ffa0d57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:53.377153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e756df-f065-4fe2-ac95-e5a61ffa0d57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:53.378765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e756df-f065-4fe2-ac95-e5a61ffa0d57 service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:53.387644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1421/2837] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:36:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:36:54.006694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf7149-343d-4961-8efb-b9b61681c30d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:54.007146 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf7149-343d-4961-8efb-b9b61681c30d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:54.007242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf7149-343d-4961-8efb-b9b61681c30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:54.028227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-70bf7149-343d-4961-8efb-b9b61681c30d None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:54.029366 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-70bf7149-343d-4961-8efb-b9b61681c30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:54.036503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1417/2838] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 14:36:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:36:54.049156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e756df-f065-4fe2-ac95-e5a61ffa0d57 service swift] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:54.060089 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e756df-f065-4fe2-ac95-e5a61ffa0d57 service swift] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:54.065166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5093dc-75f0-4cd9-9169-35f26c5b311d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:54.065764 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5093dc-75f0-4cd9-9169-35f26c5b311d service swift] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:54.066050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5093dc-75f0-4cd9-9169-35f26c5b311d service swift] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:54.066351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5093dc-75f0-4cd9-9169-35f26c5b311d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:54.074847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5093dc-75f0-4cd9-9169-35f26c5b311d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:54.076470 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5093dc-75f0-4cd9-9169-35f26c5b311d service swift] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:54.085177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1422/2839] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:36:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:36:56.734329 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf7149-343d-4961-8efb-b9b61681c30d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:56.798797 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf7149-343d-4961-8efb-b9b61681c30d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:56.803252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:56.803887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:56.804168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:56.804443 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] PATH_INFO: `/v3/users/a4649b5cfb454ea1a2fadad4cda17995` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:56.805598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4649b5cfb454ea1a2fadad4cda17995)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:56.807549 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:56.944662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4649b5cfb454ea1a2fadad4cda17995'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.023114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4649b5cfb454ea1a2fadad4cda17995'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.024054 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4649b5cfb454ea1a2fadad4cda17995'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.024700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4649b5cfb454ea1a2fadad4cda17995'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.032324 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4649b5cfb454ea1a2fadad4cda17995'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.049678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1418/2840] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:36:56 2022] DELETE /identity/v3/users/a4649b5cfb454ea1a2fadad4cda17995 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:36:57.054206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad3210-57c1-420c-80fb-e7178ef9825c None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:57.054687 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad3210-57c1-420c-80fb-e7178ef9825c None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:57.054852 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad3210-57c1-420c-80fb-e7178ef9825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:57.075061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-0aad3210-57c1-420c-80fb-e7178ef9825c None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:36:57.076846 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0aad3210-57c1-420c-80fb-e7178ef9825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:57.113665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=100 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:57.144324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1423/2841] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:36:57.154619 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:57.162583 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:57.162695 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:57.216842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c456d-13ce-478b-968b-74d6e16e65c6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:57.220663 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:57.221268 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:57.221624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:57.221898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:57.230841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:57.232290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:57.242104 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1419/2842] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:36:57.470663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad3210-57c1-420c-80fb-e7178ef9825c None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:36:57.472140 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-0aad3210-57c1-420c-80fb-e7178ef9825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:36:57.485182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad3210-57c1-420c-80fb-e7178ef9825c None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:36:57.489814 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:36:57.499927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:36:57.499927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:36:57.499927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] PATH_INFO: `/v3/projects/c690384cd06642628d5764761dc50d67` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:36:57.499927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c690384cd06642628d5764761dc50d67)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:36:57.499927 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:36:57.625460 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c690384cd06642628d5764761dc50d67', 'request_id': 'req-4512a3e5-237d-4186-b388-eeaba0ca5be0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.631728 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c690384cd06642628d5764761dc50d67', 'request_id': 'req-4512a3e5-237d-4186-b388-eeaba0ca5be0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.708802 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c690384cd06642628d5764761dc50d67', 'request_id': 'req-4512a3e5-237d-4186-b388-eeaba0ca5be0'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.720643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] The token cache is being invalidate because project c690384cd06642628d5764761dc50d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:36:57.720864 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.721494 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:36:57.723005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1424/2843] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:36:57 2022] DELETE /identity/v3/projects/c690384cd06642628d5764761dc50d67 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:37:07.359187 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=24 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:37:07.717377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:38:19.780588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:38:19.782019 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:38:19.850096 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690d5f3-86a5-418a-8034-995d11446f8e service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:38:19.910088 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:38:19.910719 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:38:19.911003 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:38:19.911273 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:38:19.913042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:38:19.924437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:38:19.926015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:38:19.936560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1420/2844] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:38:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:38:29.932320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=58 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:05.399331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:05.400825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:05.471061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512a3e5-237d-4186-b388-eeaba0ca5be0 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:05.475250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:05.475860 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:05.476135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:05.476412 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:05.478220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:05.541990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:05.543468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:05.553775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1425/2845] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:09.109814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:09.111288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:09.126204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f386ca-9c9c-4e6e-8004-d302f325e340 service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:09.133853 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:09.134505 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:09.134783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:09.135044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:09.136861 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:09.155883 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:09.158334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:09.168378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1421/2846] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:09.267543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:09.267543 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:09.280580 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4430-9681-4435-9233-919fb1b80c82 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:09.285483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:09.286117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:09.286417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:09.286685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] PATH_INFO: `/v3/users/e9925313d1d446fa81a00cb55348dff7` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:09.287791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9925313d1d446fa81a00cb55348dff7)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:09.289695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:09.516614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9925313d1d446fa81a00cb55348dff7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:09.578026 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9925313d1d446fa81a00cb55348dff7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:09.578673 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9925313d1d446fa81a00cb55348dff7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:09.591263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9925313d1d446fa81a00cb55348dff7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:09.598495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-7ab61f02-14a3-42d2-b81b-2854736ffb22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9925313d1d446fa81a00cb55348dff7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:09.604397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1426/2847] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:09 2022] DELETE /identity/v3/users/e9925313d1d446fa81a00cb55348dff7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:09.846900 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:09.858303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069e6dd-d9d9-46f2-91d8-64eaf01aedc5 service placement] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:09.917762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:09.918428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:09.918711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:09.918991 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] PATH_INFO: `/v3/projects/4c194dd986d442d3bcaa83442a49ff70` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:09.920105 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c194dd986d442d3bcaa83442a49ff70)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:09.922014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:10.024113 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c194dd986d442d3bcaa83442a49ff70', 'request_id': 'req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:10.030617 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c194dd986d442d3bcaa83442a49ff70', 'request_id': 'req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:10.110015 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c194dd986d442d3bcaa83442a49ff70', 'request_id': 'req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:10.122651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] The token cache is being invalidate because project 4c194dd986d442d3bcaa83442a49ff70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:10.122941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:10.123633 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-816f6e94-a87f-4ed2-9de5-d2fb8bf3cd6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:10.125166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1422/2848] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:09 2022] DELETE /identity/v3/projects/4c194dd986d442d3bcaa83442a49ff70 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:10.131180 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c69d-7bb9-4896-a72d-fe2ec5018f47 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.131470 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c69d-7bb9-4896-a72d-fe2ec5018f47 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.131753 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c69d-7bb9-4896-a72d-fe2ec5018f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.151258 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e682c69d-7bb9-4896-a72d-fe2ec5018f47 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:10.152732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e682c69d-7bb9-4896-a72d-fe2ec5018f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:10.169793 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:10.212054 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1427/2849] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:10.220280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24820b2-e3ba-4a7e-8296-db02ee978c34 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.220604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24820b2-e3ba-4a7e-8296-db02ee978c34 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.220910 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24820b2-e3ba-4a7e-8296-db02ee978c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.241735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-b24820b2-e3ba-4a7e-8296-db02ee978c34 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:10.242985 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b24820b2-e3ba-4a7e-8296-db02ee978c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:10.251249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1423/2850] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:10.255472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c69d-7bb9-4896-a72d-fe2ec5018f47 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:10.265293 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c69d-7bb9-4896-a72d-fe2ec5018f47 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:10.269945 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:10.272006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.272006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.272006 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.272296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:39:10.272555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:10.274161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:10.285729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1428/2851] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:39:10.290774 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24820b2-e3ba-4a7e-8296-db02ee978c34 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:10.300359 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24820b2-e3ba-4a7e-8296-db02ee978c34 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:10.304950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19431ff8-e2f8-42f4-be59-55ed001694e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:10.305931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431ff8-e2f8-42f4-be59-55ed001694e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.305931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431ff8-e2f8-42f4-be59-55ed001694e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.306256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431ff8-e2f8-42f4-be59-55ed001694e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.306708 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19431ff8-e2f8-42f4-be59-55ed001694e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:10.308603 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19431ff8-e2f8-42f4-be59-55ed001694e6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:10.385420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1424/2852] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:10 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:10.389768 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:10.400840 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a806-a20e-44b8-9af7-a5d6e5e6facc admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:10.458394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1a42-8633-4599-a96e-29dd7c44bb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:10.459620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1a42-8633-4599-a96e-29dd7c44bb91 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.459620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1a42-8633-4599-a96e-29dd7c44bb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.459620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1a42-8633-4599-a96e-29dd7c44bb91 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.462394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc1a42-8633-4599-a96e-29dd7c44bb91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:10.462672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc1a42-8633-4599-a96e-29dd7c44bb91 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:10.544813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1429/2853] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:10 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:39:10.549183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19431ff8-e2f8-42f4-be59-55ed001694e6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:10.559986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19431ff8-e2f8-42f4-be59-55ed001694e6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:10.564734 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fd27d-040f-42b7-9c62-45ae7fde78fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:10.565360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fd27d-040f-42b7-9c62-45ae7fde78fe admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.565660 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fd27d-040f-42b7-9c62-45ae7fde78fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.565903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fd27d-040f-42b7-9c62-45ae7fde78fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.566578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fd27d-040f-42b7-9c62-45ae7fde78fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:10.568177 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fd27d-040f-42b7-9c62-45ae7fde78fe admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:10.578664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1425/2854] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:39:10.582756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1a42-8633-4599-a96e-29dd7c44bb91 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:10.593733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1a42-8633-4599-a96e-29dd7c44bb91 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:10.598850 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73552f05-6193-4293-866a-2af01bca64a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:10.599455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552f05-6193-4293-866a-2af01bca64a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.599735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552f05-6193-4293-866a-2af01bca64a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.600009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552f05-6193-4293-866a-2af01bca64a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.600702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73552f05-6193-4293-866a-2af01bca64a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:10.602328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73552f05-6193-4293-866a-2af01bca64a9 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:10.614734 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1430/2855] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:10.618913 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fd27d-040f-42b7-9c62-45ae7fde78fe admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:10.628880 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fd27d-040f-42b7-9c62-45ae7fde78fe admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:10.633973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:10.634648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.634923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.635194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] PATH_INFO: `/v3/projects/2c05b7c0d85a4a04a64aa3f9b25aa9bf/users/437982b7ed6d47dd90e0232740445e69/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.650513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c05b7c0d85a4a04a64aa3f9b25aa9bf, user_id=437982b7ed6d47dd90e0232740445e69, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:10.653422 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:10.716212 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1426/2856] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:10 2022] PUT /identity/v3/projects/2c05b7c0d85a4a04a64aa3f9b25aa9bf/users/437982b7ed6d47dd90e0232740445e69/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:10.728266 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e23fe-4f75-40d9-aaa9-7a12c219c46f None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.728554 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e23fe-4f75-40d9-aaa9-7a12c219c46f None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.728829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e23fe-4f75-40d9-aaa9-7a12c219c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.756964 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-b17e23fe-4f75-40d9-aaa9-7a12c219c46f None None] MFA Rules not processed for user `437982b7ed6d47dd90e0232740445e69`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:10.758206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b17e23fe-4f75-40d9-aaa9-7a12c219c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:10.822046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1431/2857] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:10.835266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:10.837001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:10.851705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:10.910818 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f3df5-6b53-488f-ac02-baef8fec1f67 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:10.916269 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:10.916873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:10.917152 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:10.917421 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:10.926078 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:10.927925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:10.937404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1427/2858] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:39:11.030649 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e23fe-4f75-40d9-aaa9-7a12c219c46f None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:11.032157 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-b17e23fe-4f75-40d9-aaa9-7a12c219c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:11.052245 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:11.089769 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e23fe-4f75-40d9-aaa9-7a12c219c46f None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:11.093500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:11.094352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:11.094633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:11.094899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:11.096684 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:11.109890 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:11.111339 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:11.122698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1432/2859] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:11.190237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:11.191638 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:11.204109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41219a2c-7f99-4976-a226-647c59824451 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:11.207623 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22338-bb03-46d2-8a1a-3ca0a12a3417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:11.208222 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22338-bb03-46d2-8a1a-3ca0a12a3417 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:11.208590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22338-bb03-46d2-8a1a-3ca0a12a3417 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:11.209411 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22338-bb03-46d2-8a1a-3ca0a12a3417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:11.210967 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-88a22338-bb03-46d2-8a1a-3ca0a12a3417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:11.224274 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a22338-bb03-46d2-8a1a-3ca0a12a3417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:11.226275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a22338-bb03-46d2-8a1a-3ca0a12a3417 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:11.237378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1428/2860] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:11.312831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:11.377145 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a232a-9fb1-4437-8f24-4aaedc37971b service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:11.382410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:11.383041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:11.383318 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:11.383593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] PATH_INFO: `/v3/users/437982b7ed6d47dd90e0232740445e69` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:11.384716 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=437982b7ed6d47dd90e0232740445e69)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:11.386680 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:11.556296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437982b7ed6d47dd90e0232740445e69'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:11.635545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437982b7ed6d47dd90e0232740445e69'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:11.636290 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437982b7ed6d47dd90e0232740445e69'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:11.650547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437982b7ed6d47dd90e0232740445e69'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:11.657630 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437982b7ed6d47dd90e0232740445e69'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:11.663665 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1433/2861] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:11 2022] DELETE /identity/v3/users/437982b7ed6d47dd90e0232740445e69 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:11.668508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baad9-7f15-473b-803c-a18d5eae55dd None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:11.668816 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baad9-7f15-473b-803c-a18d5eae55dd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:11.669071 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baad9-7f15-473b-803c-a18d5eae55dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:11.689649 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-bb0baad9-7f15-473b-803c-a18d5eae55dd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:11.692770 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bb0baad9-7f15-473b-803c-a18d5eae55dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:11.751703 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1429/2862] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:11.761970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:11.763485 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:11.816545 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:11.835507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f6025-47a3-412d-8c00-a735f43a2c62 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:11.840479 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cb99f-d0d4-41f5-8f72-67405238c683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:11.841133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cb99f-d0d4-41f5-8f72-67405238c683 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:11.842893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cb99f-d0d4-41f5-8f72-67405238c683 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:11.842893 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cb99f-d0d4-41f5-8f72-67405238c683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:11.852386 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cb99f-d0d4-41f5-8f72-67405238c683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:11.853952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cb99f-d0d4-41f5-8f72-67405238c683 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:11.864839 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1434/2863] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:12.304456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0baad9-7f15-473b-803c-a18d5eae55dd None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:12.306110 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-bb0baad9-7f15-473b-803c-a18d5eae55dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:12.320403 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0baad9-7f15-473b-803c-a18d5eae55dd None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:12.326686 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:12.327336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:12.327854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:12.328136 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] PATH_INFO: `/v3/projects/2c05b7c0d85a4a04a64aa3f9b25aa9bf` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:12.329362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c05b7c0d85a4a04a64aa3f9b25aa9bf)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:12.331334 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:12.436736 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c05b7c0d85a4a04a64aa3f9b25aa9bf', 'request_id': 'req-b6a0da05-481a-4032-81df-6f9d08e2d802'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:12.443279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c05b7c0d85a4a04a64aa3f9b25aa9bf', 'request_id': 'req-b6a0da05-481a-4032-81df-6f9d08e2d802'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:12.519662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c05b7c0d85a4a04a64aa3f9b25aa9bf', 'request_id': 'req-b6a0da05-481a-4032-81df-6f9d08e2d802'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:12.532320 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] The token cache is being invalidate because project 2c05b7c0d85a4a04a64aa3f9b25aa9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:12.534339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:12.534339 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b6a0da05-481a-4032-81df-6f9d08e2d802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:12.534944 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1430/2864] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:12 2022] DELETE /identity/v3/projects/2c05b7c0d85a4a04a64aa3f9b25aa9bf => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:12.540557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edca8d-511f-4ce1-b93e-53500dbe32cd None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:12.541085 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edca8d-511f-4ce1-b93e-53500dbe32cd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:12.541199 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edca8d-511f-4ce1-b93e-53500dbe32cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:12.561859 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-66edca8d-511f-4ce1-b93e-53500dbe32cd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:12.563809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-66edca8d-511f-4ce1-b93e-53500dbe32cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:12.629849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1435/2865] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:12.637942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a68f1-a004-4dd3-8e00-d0288d47d711 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:12.638275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a68f1-a004-4dd3-8e00-d0288d47d711 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:12.638553 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a68f1-a004-4dd3-8e00-d0288d47d711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:12.665042 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c88a68f1-a004-4dd3-8e00-d0288d47d711 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:12.666306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c88a68f1-a004-4dd3-8e00-d0288d47d711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:12.677163 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1431/2866] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:12.681516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edca8d-511f-4ce1-b93e-53500dbe32cd None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:12.691224 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edca8d-511f-4ce1-b93e-53500dbe32cd None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:12.697547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:12.698167 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:12.698455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:12.698722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:12.699379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:39:12.699681 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:12.701280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:12.714078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1436/2867] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:39:12.719673 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a68f1-a004-4dd3-8e00-d0288d47d711 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:12.724100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:12.730239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a68f1-a004-4dd3-8e00-d0288d47d711 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:12.734979 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e3570-a13f-4f29-ae78-1a0f59dc095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:12.735578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e3570-a13f-4f29-ae78-1a0f59dc095a admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:12.735859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e3570-a13f-4f29-ae78-1a0f59dc095a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:12.740426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e3570-a13f-4f29-ae78-1a0f59dc095a admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:12.740426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e3570-a13f-4f29-ae78-1a0f59dc095a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:12.740426 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e3570-a13f-4f29-ae78-1a0f59dc095a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:12.849025 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1432/2868] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:12 2022] POST /identity/v3/projects => generated 405 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:12.853331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:12.867039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c04a9-d66b-45b7-86d9-504e95e6d98e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:12.929616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c316bb-53e0-45e7-88ad-476639391fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:12.930105 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c316bb-53e0-45e7-88ad-476639391fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:12.930418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c316bb-53e0-45e7-88ad-476639391fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:12.930695 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c316bb-53e0-45e7-88ad-476639391fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:12.931337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c316bb-53e0-45e7-88ad-476639391fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:12.933299 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c316bb-53e0-45e7-88ad-476639391fd3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:13.026596 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1437/2869] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:12 2022] POST /identity/v3/users => generated 303 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:13.031705 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e3570-a13f-4f29-ae78-1a0f59dc095a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:13.048393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e3570-a13f-4f29-ae78-1a0f59dc095a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:13.057939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37634c03-cf19-4493-b95c-1df03329e6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:13.059182 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37634c03-cf19-4493-b95c-1df03329e6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:13.059640 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37634c03-cf19-4493-b95c-1df03329e6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:13.060170 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37634c03-cf19-4493-b95c-1df03329e6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:13.061958 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37634c03-cf19-4493-b95c-1df03329e6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:13.065310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37634c03-cf19-4493-b95c-1df03329e6d6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:13.088200 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1433/2870] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:13.095194 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c316bb-53e0-45e7-88ad-476639391fd3 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:13.115451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c316bb-53e0-45e7-88ad-476639391fd3 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:13.123518 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b88c7-ec49-4d27-98f5-6de91eeee14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:13.124170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b88c7-ec49-4d27-98f5-6de91eeee14e admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:13.124456 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b88c7-ec49-4d27-98f5-6de91eeee14e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:13.124735 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b88c7-ec49-4d27-98f5-6de91eeee14e admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:13.125539 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b88c7-ec49-4d27-98f5-6de91eeee14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:13.127461 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b88c7-ec49-4d27-98f5-6de91eeee14e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:13.148708 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1438/2871] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:13.153814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37634c03-cf19-4493-b95c-1df03329e6d6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:13.165704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37634c03-cf19-4493-b95c-1df03329e6d6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:13.171086 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:13.171773 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:13.172058 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:13.172336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] PATH_INFO: `/v3/projects/d22e4dec3fa446a6a0ae43f082babbbf/users/cd093de68d834d7db77367791e542431/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:13.192375 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22e4dec3fa446a6a0ae43f082babbbf, user_id=cd093de68d834d7db77367791e542431, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:13.194168 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:13.312250 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1434/2872] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:13 2022] PUT /identity/v3/projects/d22e4dec3fa446a6a0ae43f082babbbf/users/cd093de68d834d7db77367791e542431/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:13.318225 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1543-5f16-4a46-83dd-a85fb51dc710 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:13.318557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1543-5f16-4a46-83dd-a85fb51dc710 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:13.318870 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1543-5f16-4a46-83dd-a85fb51dc710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:13.344512 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-731a1543-5f16-4a46-83dd-a85fb51dc710 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:13.346560 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-731a1543-5f16-4a46-83dd-a85fb51dc710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:13.425056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:13.449396 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1439/2873] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:13.460796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:13.463762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:13.540776 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e64ed-8683-4bf0-b0a9-0f1267c5b7ec admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:13.544461 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0e0bf-af76-4159-b41a-44c5c3a40633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:13.545062 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e0bf-af76-4159-b41a-44c5c3a40633 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:13.545338 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e0bf-af76-4159-b41a-44c5c3a40633 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:13.545611 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e0bf-af76-4159-b41a-44c5c3a40633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:13.554998 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0e0bf-af76-4159-b41a-44c5c3a40633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:13.556419 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0e0bf-af76-4159-b41a-44c5c3a40633 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:13.565962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1435/2874] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:14.861707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0088a5d-331b-4ff3-becb-af5d910670dc None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:14.862374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0088a5d-331b-4ff3-becb-af5d910670dc None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:14.862374 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0088a5d-331b-4ff3-becb-af5d910670dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:14.894541 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-e0088a5d-331b-4ff3-becb-af5d910670dc None None] MFA Rules not processed for user `cd093de68d834d7db77367791e542431`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:14.896242 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-e0088a5d-331b-4ff3-becb-af5d910670dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:14.966726 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1440/2875] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:14.971251 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0e0bf-af76-4159-b41a-44c5c3a40633 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:14.980742 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0e0bf-af76-4159-b41a-44c5c3a40633 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:14.985231 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17996d0-c4a0-4f3a-8f6d-9daad97f4939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:14.986315 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17996d0-c4a0-4f3a-8f6d-9daad97f4939 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:14.986768 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17996d0-c4a0-4f3a-8f6d-9daad97f4939 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:14.987046 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17996d0-c4a0-4f3a-8f6d-9daad97f4939 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:14.988221 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17996d0-c4a0-4f3a-8f6d-9daad97f4939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:14.990153 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17996d0-c4a0-4f3a-8f6d-9daad97f4939 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:15.122832 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1436/2876] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:14 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:39:15.127041 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0088a5d-331b-4ff3-becb-af5d910670dc None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:15.137934 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0088a5d-331b-4ff3-becb-af5d910670dc None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:15.193678 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c474a-2310-49fb-ae41-460c66eddec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:15.194222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c474a-2310-49fb-ae41-460c66eddec4 admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:15.194502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c474a-2310-49fb-ae41-460c66eddec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:15.194775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c474a-2310-49fb-ae41-460c66eddec4 admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:15.195408 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c474a-2310-49fb-ae41-460c66eddec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:15.197324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c474a-2310-49fb-ae41-460c66eddec4 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:15.308679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1441/2877] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:15 2022] POST /identity/v3/users => generated 309 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:15.312757 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17996d0-c4a0-4f3a-8f6d-9daad97f4939 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:15.323138 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17996d0-c4a0-4f3a-8f6d-9daad97f4939 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:15.327766 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b561b2a-0011-4419-aa40-9cc19cf3e603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:15.328368 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b561b2a-0011-4419-aa40-9cc19cf3e603 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:15.328651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b561b2a-0011-4419-aa40-9cc19cf3e603 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:15.328927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b561b2a-0011-4419-aa40-9cc19cf3e603 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:15.329577 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b561b2a-0011-4419-aa40-9cc19cf3e603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:15.331333 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b561b2a-0011-4419-aa40-9cc19cf3e603 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:15.341814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1437/2878] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:15.346499 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c474a-2310-49fb-ae41-460c66eddec4 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:15.357024 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c474a-2310-49fb-ae41-460c66eddec4 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:15.361585 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc1eea-2ad8-4249-a560-ac583bee06cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:15.362377 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc1eea-2ad8-4249-a560-ac583bee06cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:15.362663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc1eea-2ad8-4249-a560-ac583bee06cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:15.362938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc1eea-2ad8-4249-a560-ac583bee06cb admin admin] PATH_INFO: `/v3/projects/c46e9707be4c4a41b5d01de442cc1d8c/users/6e9006a343c54ee6b959325a43ed17a1/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:15.378031 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc1eea-2ad8-4249-a560-ac583bee06cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46e9707be4c4a41b5d01de442cc1d8c, user_id=6e9006a343c54ee6b959325a43ed17a1, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:15.380714 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc1eea-2ad8-4249-a560-ac583bee06cb admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:15.456570 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1442/2879] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:15 2022] PUT /identity/v3/projects/c46e9707be4c4a41b5d01de442cc1d8c/users/6e9006a343c54ee6b959325a43ed17a1/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:15.460467 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b561b2a-0011-4419-aa40-9cc19cf3e603 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:15.470953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b561b2a-0011-4419-aa40-9cc19cf3e603 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:15.525837 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cf2ed-c94b-41c3-9037-f825432d8528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:15.526459 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf2ed-c94b-41c3-9037-f825432d8528 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:15.526739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf2ed-c94b-41c3-9037-f825432d8528 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:15.527010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf2ed-c94b-41c3-9037-f825432d8528 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:15.527901 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cf2ed-c94b-41c3-9037-f825432d8528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:15.529473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cf2ed-c94b-41c3-9037-f825432d8528 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:15.539644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1438/2880] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:15.543824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc1eea-2ad8-4249-a560-ac583bee06cb admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:15.554038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc1eea-2ad8-4249-a560-ac583bee06cb admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:15.558775 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51203e-e2ca-46c5-8ac0-8769e3c6d1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:15.559420 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51203e-e2ca-46c5-8ac0-8769e3c6d1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:15.559727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51203e-e2ca-46c5-8ac0-8769e3c6d1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:15.560008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51203e-e2ca-46c5-8ac0-8769e3c6d1c2 admin admin] PATH_INFO: `/v3/projects/c46e9707be4c4a41b5d01de442cc1d8c/users/6e9006a343c54ee6b959325a43ed17a1/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:15.561917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51203e-e2ca-46c5-8ac0-8769e3c6d1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46e9707be4c4a41b5d01de442cc1d8c, user_id=6e9006a343c54ee6b959325a43ed17a1, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:15.564732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51203e-e2ca-46c5-8ac0-8769e3c6d1c2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:15.574886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1443/2881] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:15 2022] PUT /identity/v3/projects/c46e9707be4c4a41b5d01de442cc1d8c/users/6e9006a343c54ee6b959325a43ed17a1/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:16.758256 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf2eda-240c-4559-890c-3b68f0dadc71 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:16.758824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf2eda-240c-4559-890c-3b68f0dadc71 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:16.758824 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf2eda-240c-4559-890c-3b68f0dadc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:16.780347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:16.788632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9adf2eda-240c-4559-890c-3b68f0dadc71 None None] MFA Rules not processed for user `6e9006a343c54ee6b959325a43ed17a1`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:16.790431 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9adf2eda-240c-4559-890c-3b68f0dadc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:16.869842 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1439/2882] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:39:18.516106 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51203e-e2ca-46c5-8ac0-8769e3c6d1c2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:18.550895 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:18.582693 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51203e-e2ca-46c5-8ac0-8769e3c6d1c2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:18.587267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:18.588277 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:18.588555 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:18.588825 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] PATH_INFO: `/v3/users/cd093de68d834d7db77367791e542431` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:18.589954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd093de68d834d7db77367791e542431)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:18.591958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:18.798417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd093de68d834d7db77367791e542431'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:18.858532 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd093de68d834d7db77367791e542431'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:18.859166 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd093de68d834d7db77367791e542431'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:18.873249 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd093de68d834d7db77367791e542431'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:18.880786 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd093de68d834d7db77367791e542431'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:18.891034 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1444/2883] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:18 2022] DELETE /identity/v3/users/cd093de68d834d7db77367791e542431 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:19.150199 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf2eda-240c-4559-890c-3b68f0dadc71 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:19.162380 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf2eda-240c-4559-890c-3b68f0dadc71 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:19.227239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:19.227968 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:19.228249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:19.228520 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] PATH_INFO: `/v3/projects/d22e4dec3fa446a6a0ae43f082babbbf` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:19.229684 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22e4dec3fa446a6a0ae43f082babbbf)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:19.231653 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:19.355896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22e4dec3fa446a6a0ae43f082babbbf', 'request_id': 'req-a8575c67-8ea5-4cfe-97d2-5013ab46a907'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.362418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22e4dec3fa446a6a0ae43f082babbbf', 'request_id': 'req-a8575c67-8ea5-4cfe-97d2-5013ab46a907'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.442242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22e4dec3fa446a6a0ae43f082babbbf', 'request_id': 'req-a8575c67-8ea5-4cfe-97d2-5013ab46a907'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.454396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] The token cache is being invalidate because project d22e4dec3fa446a6a0ae43f082babbbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:19.454714 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.455362 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.457148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1440/2884] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:19 2022] DELETE /identity/v3/projects/d22e4dec3fa446a6a0ae43f082babbbf => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:19.461114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:19.462703 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:19.478637 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6914ff-a3c5-4ff8-8c74-d114d76867cc admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:19.539756 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:19.540453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:19.540653 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:19.540919 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] PATH_INFO: `/v3/users/6e9006a343c54ee6b959325a43ed17a1` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:19.542075 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9006a343c54ee6b959325a43ed17a1)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:19.544045 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:19.713352 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9006a343c54ee6b959325a43ed17a1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.717212 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:19.824431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9006a343c54ee6b959325a43ed17a1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.825155 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9006a343c54ee6b959325a43ed17a1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.839493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9006a343c54ee6b959325a43ed17a1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.846268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-a84ce89d-68aa-4679-9f90-cf85c0bed592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9006a343c54ee6b959325a43ed17a1'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:19.852849 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1445/2885] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:19 2022] DELETE /identity/v3/users/6e9006a343c54ee6b959325a43ed17a1 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:20.093400 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:20.095083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:20.118398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575c67-8ea5-4cfe-97d2-5013ab46a907 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:20.144192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:20.177920 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:20.178593 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.178870 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.179144 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] PATH_INFO: `/v3/projects/c46e9707be4c4a41b5d01de442cc1d8c` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.180288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46e9707be4c4a41b5d01de442cc1d8c)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:20.182354 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:20.319827 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46e9707be4c4a41b5d01de442cc1d8c', 'request_id': 'req-dd836562-5d5a-4c0a-a385-20f463eebe91'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:20.326481 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46e9707be4c4a41b5d01de442cc1d8c', 'request_id': 'req-dd836562-5d5a-4c0a-a385-20f463eebe91'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:20.395624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46e9707be4c4a41b5d01de442cc1d8c', 'request_id': 'req-dd836562-5d5a-4c0a-a385-20f463eebe91'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:20.407610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] The token cache is being invalidate because project c46e9707be4c4a41b5d01de442cc1d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:20.407898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:20.408529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-dd836562-5d5a-4c0a-a385-20f463eebe91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:20.410004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1441/2886] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:20 2022] DELETE /identity/v3/projects/c46e9707be4c4a41b5d01de442cc1d8c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:20.415139 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42512d2a-7713-4243-b96f-bd536c6760fa None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.415428 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42512d2a-7713-4243-b96f-bd536c6760fa None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.415702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42512d2a-7713-4243-b96f-bd536c6760fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.435524 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-42512d2a-7713-4243-b96f-bd536c6760fa None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:20.436774 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-42512d2a-7713-4243-b96f-bd536c6760fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:20.496972 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1446/2887] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:20.504566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc03b3-4536-4e16-b439-89da067c427e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.504854 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc03b3-4536-4e16-b439-89da067c427e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.505133 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc03b3-4536-4e16-b439-89da067c427e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.526188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-00fc03b3-4536-4e16-b439-89da067c427e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:20.527414 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-00fc03b3-4536-4e16-b439-89da067c427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:20.536287 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1442/2888] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:20.540645 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42512d2a-7713-4243-b96f-bd536c6760fa None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:20.550467 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42512d2a-7713-4243-b96f-bd536c6760fa None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:20.555078 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:20.555669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.555946 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.556215 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.556883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:39:20.557182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:20.558809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:20.571076 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1447/2889] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:39:20.575488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc03b3-4536-4e16-b439-89da067c427e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:20.584584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc03b3-4536-4e16-b439-89da067c427e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:20.589780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d8d9b-0311-4650-90bd-6b7257d1f2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:20.590415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8d9b-0311-4650-90bd-6b7257d1f2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.590694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8d9b-0311-4650-90bd-6b7257d1f2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.590962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8d9b-0311-4650-90bd-6b7257d1f2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.591595 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d8d9b-0311-4650-90bd-6b7257d1f2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:20.593488 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d8d9b-0311-4650-90bd-6b7257d1f2e1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:20.677211 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1443/2890] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:20 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:20.681903 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:20.691795 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551da92-9fd5-4f37-b5b5-ebae5becf037 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:20.750438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17860444-2b51-4a79-a0d2-9f14df86b47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:20.751043 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17860444-2b51-4a79-a0d2-9f14df86b47c admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.751322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17860444-2b51-4a79-a0d2-9f14df86b47c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.751593 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17860444-2b51-4a79-a0d2-9f14df86b47c admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.752222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17860444-2b51-4a79-a0d2-9f14df86b47c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:20.754130 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17860444-2b51-4a79-a0d2-9f14df86b47c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:20.812888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1448/2891] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:20 2022] POST /identity/v3/users => generated 282 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:20.817397 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d8d9b-0311-4650-90bd-6b7257d1f2e1 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:20.828456 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d8d9b-0311-4650-90bd-6b7257d1f2e1 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:20.833214 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a90ae-9d7b-4812-af92-bf0fb1b10ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:20.835458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a90ae-9d7b-4812-af92-bf0fb1b10ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.835458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a90ae-9d7b-4812-af92-bf0fb1b10ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.835458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a90ae-9d7b-4812-af92-bf0fb1b10ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.835458 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a90ae-9d7b-4812-af92-bf0fb1b10ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:20.836688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a90ae-9d7b-4812-af92-bf0fb1b10ce8 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:20.846929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1444/2892] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:20.851103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17860444-2b51-4a79-a0d2-9f14df86b47c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:20.861260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17860444-2b51-4a79-a0d2-9f14df86b47c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:20.865990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75dd83-e066-49b8-9510-4db75b185f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:20.866619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd83-e066-49b8-9510-4db75b185f44 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.867405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd83-e066-49b8-9510-4db75b185f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.867677 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd83-e066-49b8-9510-4db75b185f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.868436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75dd83-e066-49b8-9510-4db75b185f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:20.870028 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75dd83-e066-49b8-9510-4db75b185f44 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:20.880704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1449/2893] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:20.885473 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a90ae-9d7b-4812-af92-bf0fb1b10ce8 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:20.895782 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a90ae-9d7b-4812-af92-bf0fb1b10ce8 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:20.900396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460ce40-abcf-4cbb-9dc4-640b055f3569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:20.901061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460ce40-abcf-4cbb-9dc4-640b055f3569 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.901337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460ce40-abcf-4cbb-9dc4-640b055f3569 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.901605 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460ce40-abcf-4cbb-9dc4-640b055f3569 admin admin] PATH_INFO: `/v3/projects/d7b5966665ae4a0f897d55f76329b7dd/users/fb07551bbd274053ad37c912f284024d/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:20.916835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460ce40-abcf-4cbb-9dc4-640b055f3569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b5966665ae4a0f897d55f76329b7dd, user_id=fb07551bbd274053ad37c912f284024d, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:20.920889 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460ce40-abcf-4cbb-9dc4-640b055f3569 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:20.979361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1445/2894] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:20 2022] PUT /identity/v3/projects/d7b5966665ae4a0f897d55f76329b7dd/users/fb07551bbd274053ad37c912f284024d/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:20.989819 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a325aa-5959-4545-9291-985eebf3843e None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:20.990111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a325aa-5959-4545-9291-985eebf3843e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:20.990426 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a325aa-5959-4545-9291-985eebf3843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.020027 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c1a325aa-5959-4545-9291-985eebf3843e None None] MFA Rules not processed for user `fb07551bbd274053ad37c912f284024d`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:21.021267 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c1a325aa-5959-4545-9291-985eebf3843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:21.047101 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:21.090162 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1450/2895] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:21.094022 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460ce40-abcf-4cbb-9dc4-640b055f3569 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:21.152290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:21.163629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460ce40-abcf-4cbb-9dc4-640b055f3569 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:21.168949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6b862-fd90-4fae-93fe-f6e810248ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:21.169565 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6b862-fd90-4fae-93fe-f6e810248ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:21.169941 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6b862-fd90-4fae-93fe-f6e810248ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:21.170258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6b862-fd90-4fae-93fe-f6e810248ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.171032 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6b862-fd90-4fae-93fe-f6e810248ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:21.172923 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6b862-fd90-4fae-93fe-f6e810248ca3 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:21.255077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1446/2896] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:21 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:21.258755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a325aa-5959-4545-9291-985eebf3843e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:21.269994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a325aa-5959-4545-9291-985eebf3843e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:21.338117 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542187f-0751-44ed-bfe0-6599aa7c4f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:21.339327 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542187f-0751-44ed-bfe0-6599aa7c4f8b admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:21.339616 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542187f-0751-44ed-bfe0-6599aa7c4f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:21.339897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542187f-0751-44ed-bfe0-6599aa7c4f8b admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.340599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542187f-0751-44ed-bfe0-6599aa7c4f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:21.342757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542187f-0751-44ed-bfe0-6599aa7c4f8b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:21.436505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1451/2897] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:21 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:21.440629 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6b862-fd90-4fae-93fe-f6e810248ca3 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:21.451555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6b862-fd90-4fae-93fe-f6e810248ca3 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:21.456672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092fddd-db01-4646-a046-7b6c32417e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:21.457282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092fddd-db01-4646-a046-7b6c32417e43 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:21.457558 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092fddd-db01-4646-a046-7b6c32417e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:21.457830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092fddd-db01-4646-a046-7b6c32417e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.458508 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092fddd-db01-4646-a046-7b6c32417e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:21.460205 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092fddd-db01-4646-a046-7b6c32417e43 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:21.471688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1447/2898] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:21.476147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542187f-0751-44ed-bfe0-6599aa7c4f8b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:21.491826 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542187f-0751-44ed-bfe0-6599aa7c4f8b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:21.496702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075fb9d-75a9-42f9-8207-143f673b9ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:21.497387 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fb9d-75a9-42f9-8207-143f673b9ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:21.497664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fb9d-75a9-42f9-8207-143f673b9ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:21.497942 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fb9d-75a9-42f9-8207-143f673b9ec7 admin admin] PATH_INFO: `/v3/projects/4fb35e6228ca4d52b5524743ff35cca5/users/1095647c4a454e6d8d1bfc5a69323b5c/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.513748 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075fb9d-75a9-42f9-8207-143f673b9ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb35e6228ca4d52b5524743ff35cca5, user_id=1095647c4a454e6d8d1bfc5a69323b5c, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:21.516508 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075fb9d-75a9-42f9-8207-143f673b9ec7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:21.611537 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1452/2899] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:21 2022] PUT /identity/v3/projects/4fb35e6228ca4d52b5524743ff35cca5/users/1095647c4a454e6d8d1bfc5a69323b5c/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:21.615716 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092fddd-db01-4646-a046-7b6c32417e43 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:21.626089 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092fddd-db01-4646-a046-7b6c32417e43 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:21.688702 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304ac09-65a5-41f1-ab72-bffbb786d398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:21.689325 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304ac09-65a5-41f1-ab72-bffbb786d398 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:21.689677 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304ac09-65a5-41f1-ab72-bffbb786d398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:21.689896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304ac09-65a5-41f1-ab72-bffbb786d398 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.690643 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304ac09-65a5-41f1-ab72-bffbb786d398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:21.692293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304ac09-65a5-41f1-ab72-bffbb786d398 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:21.702779 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1448/2900] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:21.707047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075fb9d-75a9-42f9-8207-143f673b9ec7 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:21.718561 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075fb9d-75a9-42f9-8207-143f673b9ec7 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:21.723780 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:21.724452 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:21.724732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:21.725009 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] PATH_INFO: `/v3/projects/4fb35e6228ca4d52b5524743ff35cca5/users/1095647c4a454e6d8d1bfc5a69323b5c/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.729226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb35e6228ca4d52b5524743ff35cca5, user_id=1095647c4a454e6d8d1bfc5a69323b5c, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:21.731989 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:21.743113 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1453/2901] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:21 2022] PUT /identity/v3/projects/4fb35e6228ca4d52b5524743ff35cca5/users/1095647c4a454e6d8d1bfc5a69323b5c/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:39:21.763829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450fc8-c5f7-4615-8b49-915def61107c None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:21.764183 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450fc8-c5f7-4615-8b49-915def61107c None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:21.764465 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450fc8-c5f7-4615-8b49-915def61107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.796903 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-9c450fc8-c5f7-4615-8b49-915def61107c None None] MFA Rules not processed for user `1095647c4a454e6d8d1bfc5a69323b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:21.798181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-9c450fc8-c5f7-4615-8b49-915def61107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:21.874762 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1449/2902] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:21.887754 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:21.889496 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:21.951150 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3ce8f-162c-41bb-9302-978f78a0a75f admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:21.956188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91fa43-d2e1-4213-9333-6d88bbb8baff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:21.956813 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91fa43-d2e1-4213-9333-6d88bbb8baff service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:21.957091 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91fa43-d2e1-4213-9333-6d88bbb8baff service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:21.957403 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91fa43-d2e1-4213-9333-6d88bbb8baff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:21.968500 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91fa43-d2e1-4213-9333-6d88bbb8baff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:21.969995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91fa43-d2e1-4213-9333-6d88bbb8baff service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:21.980372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1454/2903] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:22.227969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c450fc8-c5f7-4615-8b49-915def61107c None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:22.239490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c450fc8-c5f7-4615-8b49-915def61107c None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:22.245166 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b18fed-f476-4b38-a4a3-f5cd391cac52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:22.245783 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b18fed-f476-4b38-a4a3-f5cd391cac52 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:22.246068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b18fed-f476-4b38-a4a3-f5cd391cac52 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:22.246378 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b18fed-f476-4b38-a4a3-f5cd391cac52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:22.295150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:22.307515 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b18fed-f476-4b38-a4a3-f5cd391cac52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:22.308961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b18fed-f476-4b38-a4a3-f5cd391cac52 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:22.318977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1450/2904] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:39:25.826022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91fa43-d2e1-4213-9333-6d88bbb8baff service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:25.847818 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:25.890958 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91fa43-d2e1-4213-9333-6d88bbb8baff service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:25.895622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:25.896453 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:25.896944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:25.897255 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] PATH_INFO: `/v3/users/fb07551bbd274053ad37c912f284024d` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:25.898669 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb07551bbd274053ad37c912f284024d)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:25.900627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:26.052858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb07551bbd274053ad37c912f284024d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.132263 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb07551bbd274053ad37c912f284024d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.132883 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb07551bbd274053ad37c912f284024d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.145990 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb07551bbd274053ad37c912f284024d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.153606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb07551bbd274053ad37c912f284024d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.159599 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1455/2905] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:25 2022] DELETE /identity/v3/users/fb07551bbd274053ad37c912f284024d => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:26.164207 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1e36-98f2-4f8d-9c19-7cfba60368cd None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:26.164657 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1e36-98f2-4f8d-9c19-7cfba60368cd None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:26.164780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1e36-98f2-4f8d-9c19-7cfba60368cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:26.184966 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-e06c1e36-98f2-4f8d-9c19-7cfba60368cd None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:26.186184 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e06c1e36-98f2-4f8d-9c19-7cfba60368cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:26.247180 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1451/2906] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:26.257161 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:26.258613 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:26.316727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971dee9-fb4e-4827-96cb-9056f2b60df1 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:26.320446 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b996e-d568-4500-ad77-06b65160aa10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:26.321058 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b996e-d568-4500-ad77-06b65160aa10 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:26.321337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b996e-d568-4500-ad77-06b65160aa10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:26.321606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b996e-d568-4500-ad77-06b65160aa10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:26.330783 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b996e-d568-4500-ad77-06b65160aa10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:26.332268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b996e-d568-4500-ad77-06b65160aa10 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:26.342156 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1456/2907] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:26.733047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c1e36-98f2-4f8d-9c19-7cfba60368cd None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:26.734613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-e06c1e36-98f2-4f8d-9c19-7cfba60368cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:26.746928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c1e36-98f2-4f8d-9c19-7cfba60368cd None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:26.751916 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:26.752631 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:26.752902 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:26.753169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] PATH_INFO: `/v3/projects/d7b5966665ae4a0f897d55f76329b7dd` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:26.754312 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7b5966665ae4a0f897d55f76329b7dd)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:26.756252 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:26.859303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b5966665ae4a0f897d55f76329b7dd', 'request_id': 'req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.865639 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b5966665ae4a0f897d55f76329b7dd', 'request_id': 'req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.942381 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b5966665ae4a0f897d55f76329b7dd', 'request_id': 'req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.954070 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] The token cache is being invalidate because project d7b5966665ae4a0f897d55f76329b7dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:26.954393 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.955122 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:26.956622 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1452/2908] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:26 2022] DELETE /identity/v3/projects/d7b5966665ae4a0f897d55f76329b7dd => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:26.960747 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b996e-d568-4500-ad77-06b65160aa10 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:26.970808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b996e-d568-4500-ad77-06b65160aa10 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:27.015791 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:27.031858 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:27.032516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:27.032796 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:27.033074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] PATH_INFO: `/v3/users/1095647c4a454e6d8d1bfc5a69323b5c` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:27.034271 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1095647c4a454e6d8d1bfc5a69323b5c)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:27.036218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:27.203274 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1095647c4a454e6d8d1bfc5a69323b5c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:27.280389 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1095647c4a454e6d8d1bfc5a69323b5c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:27.280998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1095647c4a454e6d8d1bfc5a69323b5c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:27.294992 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1095647c4a454e6d8d1bfc5a69323b5c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:27.302581 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-e770c9dc-5453-44c3-9bb1-644a5064f2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1095647c4a454e6d8d1bfc5a69323b5c'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:27.309544 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1457/2909] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:26 2022] DELETE /identity/v3/users/1095647c4a454e6d8d1bfc5a69323b5c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:27.700065 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:27.701650 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:27.715238 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcb72d-00a2-4cf4-81c0-dbaf2a1d6d61 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:27.774074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:27.774805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:27.775083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:27.776029 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] PATH_INFO: `/v3/projects/4fb35e6228ca4d52b5524743ff35cca5` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:27.777255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb35e6228ca4d52b5524743ff35cca5)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:27.779627 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:27.792043 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:27.899217 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb35e6228ca4d52b5524743ff35cca5', 'request_id': 'req-b557e067-f53e-41fc-8795-206bfbc17d0b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:27.906346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb35e6228ca4d52b5524743ff35cca5', 'request_id': 'req-b557e067-f53e-41fc-8795-206bfbc17d0b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:28.006713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb35e6228ca4d52b5524743ff35cca5', 'request_id': 'req-b557e067-f53e-41fc-8795-206bfbc17d0b'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:28.018925 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] The token cache is being invalidate because project 4fb35e6228ca4d52b5524743ff35cca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:28.022340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:28.022340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b557e067-f53e-41fc-8795-206bfbc17d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:28.022340 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1453/2910] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:27 2022] DELETE /identity/v3/projects/4fb35e6228ca4d52b5524743ff35cca5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:28.027124 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd897d-7e6f-4690-a50b-51df9d4210fa None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.027607 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd897d-7e6f-4690-a50b-51df9d4210fa None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.027739 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd897d-7e6f-4690-a50b-51df9d4210fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.051025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-5cfd897d-7e6f-4690-a50b-51df9d4210fa None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:28.051025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-5cfd897d-7e6f-4690-a50b-51df9d4210fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:28.108052 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1458/2911] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:28.115982 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff40be-b83d-4b02-8919-dc6477ff86d8 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.116275 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff40be-b83d-4b02-8919-dc6477ff86d8 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.116576 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff40be-b83d-4b02-8919-dc6477ff86d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.138061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f9ff40be-b83d-4b02-8919-dc6477ff86d8 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:28.139349 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f9ff40be-b83d-4b02-8919-dc6477ff86d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:28.147424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1454/2912] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:28.151622 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd897d-7e6f-4690-a50b-51df9d4210fa None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:28.161949 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd897d-7e6f-4690-a50b-51df9d4210fa None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:28.166643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:28.167250 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.167533 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.167808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.168457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:39:28.168761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:28.170384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:28.182477 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1459/2913] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:39:28.186758 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff40be-b83d-4b02-8919-dc6477ff86d8 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:28.195896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff40be-b83d-4b02-8919-dc6477ff86d8 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:28.201331 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eca62-9224-477d-a69f-63239312e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:28.201934 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eca62-9224-477d-a69f-63239312e8fd admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.202249 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eca62-9224-477d-a69f-63239312e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.202525 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eca62-9224-477d-a69f-63239312e8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.203167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376eca62-9224-477d-a69f-63239312e8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:28.205103 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376eca62-9224-477d-a69f-63239312e8fd admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:28.312485 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1455/2914] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:28 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:28.317720 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:28.328672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184220a5-d2cd-4adc-8534-247af5e6f3aa admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:28.372994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:28.384316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756241a-8933-4dbc-9f95-fbd029933d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:28.385563 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756241a-8933-4dbc-9f95-fbd029933d4d admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.385847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756241a-8933-4dbc-9f95-fbd029933d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.386121 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756241a-8933-4dbc-9f95-fbd029933d4d admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.386781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756241a-8933-4dbc-9f95-fbd029933d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:28.388823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756241a-8933-4dbc-9f95-fbd029933d4d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:28.447940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1460/2915] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:28 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:28.452204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eca62-9224-477d-a69f-63239312e8fd admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:28.463074 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eca62-9224-477d-a69f-63239312e8fd admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:28.467704 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0bc49-4417-46fb-b1c9-0a1765521bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:28.468332 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0bc49-4417-46fb-b1c9-0a1765521bed admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.468610 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0bc49-4417-46fb-b1c9-0a1765521bed admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.468881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0bc49-4417-46fb-b1c9-0a1765521bed admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.469529 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0bc49-4417-46fb-b1c9-0a1765521bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:28.471140 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0bc49-4417-46fb-b1c9-0a1765521bed admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:28.481654 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1456/2916] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:28.485690 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756241a-8933-4dbc-9f95-fbd029933d4d admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:28.496020 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756241a-8933-4dbc-9f95-fbd029933d4d admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:28.500526 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e266944-fe9d-4b94-a9f1-47154a05a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:28.501107 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266944-fe9d-4b94-a9f1-47154a05a59b admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.501384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266944-fe9d-4b94-a9f1-47154a05a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.501654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266944-fe9d-4b94-a9f1-47154a05a59b admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.502324 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e266944-fe9d-4b94-a9f1-47154a05a59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:28.503914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e266944-fe9d-4b94-a9f1-47154a05a59b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:28.514447 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1461/2917] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:28.518510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0bc49-4417-46fb-b1c9-0a1765521bed admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:28.528822 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0bc49-4417-46fb-b1c9-0a1765521bed admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:28.533814 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46902c7a-eb2b-46c7-957c-ced75cafb963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:28.534503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902c7a-eb2b-46c7-957c-ced75cafb963 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.534784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902c7a-eb2b-46c7-957c-ced75cafb963 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.535059 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902c7a-eb2b-46c7-957c-ced75cafb963 admin admin] PATH_INFO: `/v3/projects/fc6c3c7004e640688377fd0a52f06b1f/users/80346a8d830145088e6a9805d189d206/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.550185 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46902c7a-eb2b-46c7-957c-ced75cafb963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6c3c7004e640688377fd0a52f06b1f, user_id=80346a8d830145088e6a9805d189d206, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:28.552904 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46902c7a-eb2b-46c7-957c-ced75cafb963 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:28.659489 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1457/2918] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:28 2022] PUT /identity/v3/projects/fc6c3c7004e640688377fd0a52f06b1f/users/80346a8d830145088e6a9805d189d206/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:39:28.671368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c663e9a-362e-4edb-8e64-05c4e9473353 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.671654 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c663e9a-362e-4edb-8e64-05c4e9473353 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.671952 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c663e9a-362e-4edb-8e64-05c4e9473353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.699845 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-3c663e9a-362e-4edb-8e64-05c4e9473353 None None] MFA Rules not processed for user `80346a8d830145088e6a9805d189d206`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:28.701064 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-3c663e9a-362e-4edb-8e64-05c4e9473353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:28.764340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1462/2919] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:28.768872 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46902c7a-eb2b-46c7-957c-ced75cafb963 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:28.831208 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46902c7a-eb2b-46c7-957c-ced75cafb963 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:28.835737 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8644-1280-4887-9b42-525af14d83db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:28.836322 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8644-1280-4887-9b42-525af14d83db admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:28.836599 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8644-1280-4887-9b42-525af14d83db admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:28.836868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8644-1280-4887-9b42-525af14d83db admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:28.837490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8644-1280-4887-9b42-525af14d83db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:28.839905 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8644-1280-4887-9b42-525af14d83db admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:28.927984 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1458/2920] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:28 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:28.932039 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c663e9a-362e-4edb-8e64-05c4e9473353 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:28.942957 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c663e9a-362e-4edb-8e64-05c4e9473353 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:28.999337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed74819-544c-4d68-ab1b-30ad3af39e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:28.999940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed74819-544c-4d68-ab1b-30ad3af39e4e admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:29.000222 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed74819-544c-4d68-ab1b-30ad3af39e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:29.000495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed74819-544c-4d68-ab1b-30ad3af39e4e admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:29.001127 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed74819-544c-4d68-ab1b-30ad3af39e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:29.003105 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed74819-544c-4d68-ab1b-30ad3af39e4e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:29.110054 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1463/2921] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:28 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:29.114100 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8644-1280-4887-9b42-525af14d83db admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:29.124648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8644-1280-4887-9b42-525af14d83db admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:29.129343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1179832-5fc5-4212-941f-437e6317f883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:29.129945 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1179832-5fc5-4212-941f-437e6317f883 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:29.130279 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1179832-5fc5-4212-941f-437e6317f883 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:29.130540 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1179832-5fc5-4212-941f-437e6317f883 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:29.131295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1179832-5fc5-4212-941f-437e6317f883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:29.132896 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1179832-5fc5-4212-941f-437e6317f883 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:29.144120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1459/2922] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:29.148068 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed74819-544c-4d68-ab1b-30ad3af39e4e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:29.159189 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed74819-544c-4d68-ab1b-30ad3af39e4e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:29.163838 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc9d46-3f93-42fe-b220-72f197b8b26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:29.164488 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc9d46-3f93-42fe-b220-72f197b8b26c admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:29.164770 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc9d46-3f93-42fe-b220-72f197b8b26c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:29.165046 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc9d46-3f93-42fe-b220-72f197b8b26c admin admin] PATH_INFO: `/v3/projects/d3c9ead9b32e490cab84951a509b0119/users/f5a42a45fff24933afd63e2633359f83/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:29.180953 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc9d46-3f93-42fe-b220-72f197b8b26c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c9ead9b32e490cab84951a509b0119, user_id=f5a42a45fff24933afd63e2633359f83, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:29.183626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc9d46-3f93-42fe-b220-72f197b8b26c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:29.269959 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1464/2923] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:29 2022] PUT /identity/v3/projects/d3c9ead9b32e490cab84951a509b0119/users/f5a42a45fff24933afd63e2633359f83/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:29.274933 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1179832-5fc5-4212-941f-437e6317f883 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:29.285849 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1179832-5fc5-4212-941f-437e6317f883 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:29.293586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:29.341798 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650075a-0eaf-444f-ad06-dfe0b914eb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:29.342440 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650075a-0eaf-444f-ad06-dfe0b914eb5d admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:29.342717 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650075a-0eaf-444f-ad06-dfe0b914eb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:29.342990 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650075a-0eaf-444f-ad06-dfe0b914eb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:29.343624 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650075a-0eaf-444f-ad06-dfe0b914eb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:29.345209 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650075a-0eaf-444f-ad06-dfe0b914eb5d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:29.356314 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1460/2924] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:29.360520 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc9d46-3f93-42fe-b220-72f197b8b26c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:29.370954 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc9d46-3f93-42fe-b220-72f197b8b26c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:29.375644 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:29.377227 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:29.377227 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:29.377227 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] PATH_INFO: `/v3/projects/d3c9ead9b32e490cab84951a509b0119/users/f5a42a45fff24933afd63e2633359f83/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:29.378909 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c9ead9b32e490cab84951a509b0119, user_id=f5a42a45fff24933afd63e2633359f83, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:29.381604 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:29.393135 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1465/2925] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:29 2022] PUT /identity/v3/projects/d3c9ead9b32e490cab84951a509b0119/users/f5a42a45fff24933afd63e2633359f83/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:29.404261 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df83dbb-653d-4256-99c3-84552724ce16 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:29.404551 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df83dbb-653d-4256-99c3-84552724ce16 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:29.404829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df83dbb-653d-4256-99c3-84552724ce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:29.432408 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-1df83dbb-653d-4256-99c3-84552724ce16 None None] MFA Rules not processed for user `f5a42a45fff24933afd63e2633359f83`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:29.433560 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1df83dbb-653d-4256-99c3-84552724ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:29.502613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1461/2926] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:39:29.513847 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:29.515619 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:29.526974 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:29.573481 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f951d2d-aa02-4a79-a56c-c96bd5d078ee admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:29.577182 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d9ad9-74a2-4b9e-954b-9d45771017ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:29.577781 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d9ad9-74a2-4b9e-954b-9d45771017ba service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:29.578056 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d9ad9-74a2-4b9e-954b-9d45771017ba service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:29.578351 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d9ad9-74a2-4b9e-954b-9d45771017ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:29.587081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d9ad9-74a2-4b9e-954b-9d45771017ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:29.589095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d9ad9-74a2-4b9e-954b-9d45771017ba service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:29.598914 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1466/2927] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:29.708539 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df83dbb-653d-4256-99c3-84552724ce16 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:29.718343 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df83dbb-653d-4256-99c3-84552724ce16 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:29.722216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9d606-245e-499c-a67c-a1ce14db3870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:29.722893 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9d606-245e-499c-a67c-a1ce14db3870 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:29.723237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9d606-245e-499c-a67c-a1ce14db3870 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:29.723519 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9d606-245e-499c-a67c-a1ce14db3870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:29.780755 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9d606-245e-499c-a67c-a1ce14db3870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:29.782229 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9d606-245e-499c-a67c-a1ce14db3870 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:29.791189 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1462/2928] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:32.888991 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d9ad9-74a2-4b9e-954b-9d45771017ba service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:32.949307 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d9ad9-74a2-4b9e-954b-9d45771017ba service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:32.953820 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:32.954475 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:32.954755 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:32.955025 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] PATH_INFO: `/v3/users/80346a8d830145088e6a9805d189d206` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:32.956128 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80346a8d830145088e6a9805d189d206)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:32.958042 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:33.134440 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80346a8d830145088e6a9805d189d206'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:33.178472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80346a8d830145088e6a9805d189d206'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:33.179103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80346a8d830145088e6a9805d189d206'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:33.192594 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80346a8d830145088e6a9805d189d206'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:33.199778 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80346a8d830145088e6a9805d189d206'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:33.205757 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1467/2929] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:32 2022] DELETE /identity/v3/users/80346a8d830145088e6a9805d189d206 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:33.210284 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98bb6-0638-41b1-92ea-3c978f53da82 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:33.210735 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98bb6-0638-41b1-92ea-3c978f53da82 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:33.210845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98bb6-0638-41b1-92ea-3c978f53da82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:33.230047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-64e98bb6-0638-41b1-92ea-3c978f53da82 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:33.231286 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-64e98bb6-0638-41b1-92ea-3c978f53da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:33.256632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=100 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:33.289213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1463/2930] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:33.299529 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:33.300970 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:33.342730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:33.366547 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d63b95-e6e3-4b4f-9f3e-b518c4665498 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:33.370589 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef3dc7-9af8-498c-9738-ece79c703cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:33.371193 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef3dc7-9af8-498c-9738-ece79c703cfd service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:33.371468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef3dc7-9af8-498c-9738-ece79c703cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:33.371738 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef3dc7-9af8-498c-9738-ece79c703cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:33.380906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef3dc7-9af8-498c-9738-ece79c703cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:33.382397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef3dc7-9af8-498c-9738-ece79c703cfd service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:33.391709 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1468/2931] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:33.837049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e98bb6-0638-41b1-92ea-3c978f53da82 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:33.839193 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-64e98bb6-0638-41b1-92ea-3c978f53da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:33.851396 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e98bb6-0638-41b1-92ea-3c978f53da82 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:33.856140 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:33.856775 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:33.857057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:33.857336 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] PATH_INFO: `/v3/projects/fc6c3c7004e640688377fd0a52f06b1f` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:33.858566 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc6c3c7004e640688377fd0a52f06b1f)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:33.860546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:33.967661 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc6c3c7004e640688377fd0a52f06b1f', 'request_id': 'req-5865ba73-17d4-4914-a0d5-da760fcf7e7a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:33.973799 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc6c3c7004e640688377fd0a52f06b1f', 'request_id': 'req-5865ba73-17d4-4914-a0d5-da760fcf7e7a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.035470 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc6c3c7004e640688377fd0a52f06b1f', 'request_id': 'req-5865ba73-17d4-4914-a0d5-da760fcf7e7a'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.046363 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] The token cache is being invalidate because project fc6c3c7004e640688377fd0a52f06b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:34.046662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.047288 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.048875 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1464/2932] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:33 2022] DELETE /identity/v3/projects/fc6c3c7004e640688377fd0a52f06b1f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:34.052910 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef3dc7-9af8-498c-9738-ece79c703cfd service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:34.062659 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef3dc7-9af8-498c-9738-ece79c703cfd service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:34.118229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:34.119394 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:34.119679 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:34.119986 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] PATH_INFO: `/v3/users/f5a42a45fff24933afd63e2633359f83` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:34.121120 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a42a45fff24933afd63e2633359f83)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:34.123098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:34.323863 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a42a45fff24933afd63e2633359f83'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.389729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a42a45fff24933afd63e2633359f83'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.390326 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a42a45fff24933afd63e2633359f83'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.402701 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a42a45fff24933afd63e2633359f83'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.409733 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-222691e1-d84b-4ee6-bf54-426f7d32da45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a42a45fff24933afd63e2633359f83'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:34.415633 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1469/2933] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:34 2022] DELETE /identity/v3/users/f5a42a45fff24933afd63e2633359f83 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:34.831335 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:34.832784 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:34.848761 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865ba73-17d4-4914-a0d5-da760fcf7e7a admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:34.909449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:34.909995 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:34.910310 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:34.910585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] PATH_INFO: `/v3/projects/d3c9ead9b32e490cab84951a509b0119` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:34.911692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c9ead9b32e490cab84951a509b0119)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:34.913594 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:35.024651 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c9ead9b32e490cab84951a509b0119', 'request_id': 'req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:35.031499 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c9ead9b32e490cab84951a509b0119', 'request_id': 'req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:35.034007 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:35.138953 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c9ead9b32e490cab84951a509b0119', 'request_id': 'req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:35.150859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] The token cache is being invalidate because project d3c9ead9b32e490cab84951a509b0119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:35.151147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:35.151780 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-5ee998e9-f53c-460e-85e7-39a6f3cbf40f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:35.153533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1465/2934] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:34 2022] DELETE /identity/v3/projects/d3c9ead9b32e490cab84951a509b0119 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:35.158625 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79ef11-e5ab-4a95-b993-2dbcffd3303e None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.159133 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79ef11-e5ab-4a95-b993-2dbcffd3303e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.159252 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79ef11-e5ab-4a95-b993-2dbcffd3303e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.179179 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-6c79ef11-e5ab-4a95-b993-2dbcffd3303e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:35.180312 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6c79ef11-e5ab-4a95-b993-2dbcffd3303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:35.228306 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:35.239993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1470/2935] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:35.247838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccec9a4-4ffc-4384-a1e3-f13eaea6b403 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.248124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccec9a4-4ffc-4384-a1e3-f13eaea6b403 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.248398 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccec9a4-4ffc-4384-a1e3-f13eaea6b403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.268415 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-2ccec9a4-4ffc-4384-a1e3-f13eaea6b403 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:35.269662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-2ccec9a4-4ffc-4384-a1e3-f13eaea6b403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:35.278064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1466/2936] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:35.282549 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79ef11-e5ab-4a95-b993-2dbcffd3303e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:35.292022 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79ef11-e5ab-4a95-b993-2dbcffd3303e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:35.296579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:35.297177 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.300451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.300451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] PATH_INFO: `/v3/domains` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.300451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:39:35.300451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:35.300451 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:35.311597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1471/2937] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:39:35.315994 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccec9a4-4ffc-4384-a1e3-f13eaea6b403 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:35.325956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccec9a4-4ffc-4384-a1e3-f13eaea6b403 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:35.330689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046db6-6e03-45a8-b210-0e5b0082f51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:35.331293 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046db6-6e03-45a8-b210-0e5b0082f51e admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.331573 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046db6-6e03-45a8-b210-0e5b0082f51e admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.331846 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046db6-6e03-45a8-b210-0e5b0082f51e admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.332495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4046db6-6e03-45a8-b210-0e5b0082f51e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:35.334423 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4046db6-6e03-45a8-b210-0e5b0082f51e admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:35.418068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1467/2938] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:35 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:35.422147 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:35.432213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25173-cc90-49ff-9679-0ef1d259f276 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:35.488997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c556c9-94b1-43da-b9ba-4e00faa9f33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:35.489597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c556c9-94b1-43da-b9ba-4e00faa9f33c admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.489872 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c556c9-94b1-43da-b9ba-4e00faa9f33c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.490168 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c556c9-94b1-43da-b9ba-4e00faa9f33c admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.490794 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c556c9-94b1-43da-b9ba-4e00faa9f33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:35.492730 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c556c9-94b1-43da-b9ba-4e00faa9f33c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:35.572509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1472/2939] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:35 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:35.577258 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046db6-6e03-45a8-b210-0e5b0082f51e admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:35.587664 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046db6-6e03-45a8-b210-0e5b0082f51e admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:35.592382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f69cb4-7243-487c-9e9b-094a884fb113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:35.592956 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f69cb4-7243-487c-9e9b-094a884fb113 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.593237 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f69cb4-7243-487c-9e9b-094a884fb113 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.593512 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f69cb4-7243-487c-9e9b-094a884fb113 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.598942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f69cb4-7243-487c-9e9b-094a884fb113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:35.598942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f69cb4-7243-487c-9e9b-094a884fb113 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:35.605897 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1468/2940] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:35.609908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c556c9-94b1-43da-b9ba-4e00faa9f33c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:35.620808 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c556c9-94b1-43da-b9ba-4e00faa9f33c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:35.625337 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13222a9a-4ea8-4bab-b4bf-0543b48f1188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:35.625973 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13222a9a-4ea8-4bab-b4bf-0543b48f1188 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.626276 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13222a9a-4ea8-4bab-b4bf-0543b48f1188 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.626553 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13222a9a-4ea8-4bab-b4bf-0543b48f1188 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.627185 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13222a9a-4ea8-4bab-b4bf-0543b48f1188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:35.628765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13222a9a-4ea8-4bab-b4bf-0543b48f1188 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:35.640363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1473/2941] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:35.644588 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f69cb4-7243-487c-9e9b-094a884fb113 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:35.654836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f69cb4-7243-487c-9e9b-094a884fb113 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:35.659510 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902cd70-56ca-4de7-acfe-d92a4b84ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:35.660148 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902cd70-56ca-4de7-acfe-d92a4b84ea37 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.660424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902cd70-56ca-4de7-acfe-d92a4b84ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.660694 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902cd70-56ca-4de7-acfe-d92a4b84ea37 admin admin] PATH_INFO: `/v3/projects/41599d2bcea240d78c26d0f262cde0bf/users/a60e6bb2b1da4de89ec1f3d1db82d973/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.675778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902cd70-56ca-4de7-acfe-d92a4b84ea37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41599d2bcea240d78c26d0f262cde0bf, user_id=a60e6bb2b1da4de89ec1f3d1db82d973, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:35.678430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902cd70-56ca-4de7-acfe-d92a4b84ea37 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:35.743667 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1469/2942] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:35 2022] PUT /identity/v3/projects/41599d2bcea240d78c26d0f262cde0bf/users/a60e6bb2b1da4de89ec1f3d1db82d973/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:35.755745 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68c40e-1f74-4e4a-9574-ddcfa223dc70 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.756030 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68c40e-1f74-4e4a-9574-ddcfa223dc70 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.756302 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68c40e-1f74-4e4a-9574-ddcfa223dc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.784260 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-cf68c40e-1f74-4e4a-9574-ddcfa223dc70 None None] MFA Rules not processed for user `a60e6bb2b1da4de89ec1f3d1db82d973`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:35.785458 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-cf68c40e-1f74-4e4a-9574-ddcfa223dc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:35.847360 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1474/2943] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:35.851908 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902cd70-56ca-4de7-acfe-d92a4b84ea37 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:35.913580 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902cd70-56ca-4de7-acfe-d92a4b84ea37 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:35.918281 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce108-803d-45db-9794-de042e74edde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:35.918887 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce108-803d-45db-9794-de042e74edde admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:35.919169 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce108-803d-45db-9794-de042e74edde admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:35.919444 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce108-803d-45db-9794-de042e74edde admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:35.920072 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce108-803d-45db-9794-de042e74edde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:35.921975 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce108-803d-45db-9794-de042e74edde admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.025772 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1470/2944] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:35 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:36.030008 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68c40e-1f74-4e4a-9574-ddcfa223dc70 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.040867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68c40e-1f74-4e4a-9574-ddcfa223dc70 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:36.085296 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:36.096804 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11adea-10f5-4bda-81b1-3ef83d50e4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:36.097505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11adea-10f5-4bda-81b1-3ef83d50e4dc admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.097788 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11adea-10f5-4bda-81b1-3ef83d50e4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.098062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11adea-10f5-4bda-81b1-3ef83d50e4dc admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.098727 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11adea-10f5-4bda-81b1-3ef83d50e4dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:36.100761 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11adea-10f5-4bda-81b1-3ef83d50e4dc admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.161268 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1475/2945] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:36 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:36.165266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce108-803d-45db-9794-de042e74edde admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.176171 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce108-803d-45db-9794-de042e74edde admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:36.180907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa309c-9384-4c1c-bd0d-6ce4db7cb2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:36.181548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa309c-9384-4c1c-bd0d-6ce4db7cb2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.181830 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa309c-9384-4c1c-bd0d-6ce4db7cb2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.182102 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa309c-9384-4c1c-bd0d-6ce4db7cb2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.182866 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa309c-9384-4c1c-bd0d-6ce4db7cb2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:36.185049 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa309c-9384-4c1c-bd0d-6ce4db7cb2b6 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.195648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1471/2946] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:36.199565 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11adea-10f5-4bda-81b1-3ef83d50e4dc admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.210220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11adea-10f5-4bda-81b1-3ef83d50e4dc admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:36.214835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42324abf-e933-42c3-a5e2-82d028bfd413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:36.215410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42324abf-e933-42c3-a5e2-82d028bfd413 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.215692 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42324abf-e933-42c3-a5e2-82d028bfd413 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.215968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42324abf-e933-42c3-a5e2-82d028bfd413 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.216605 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42324abf-e933-42c3-a5e2-82d028bfd413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:36.218230 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42324abf-e933-42c3-a5e2-82d028bfd413 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.229229 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1476/2947] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:36.234012 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa309c-9384-4c1c-bd0d-6ce4db7cb2b6 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.249418 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa309c-9384-4c1c-bd0d-6ce4db7cb2b6 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:36.256119 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69978a-cff3-43f7-b89d-f09acf041241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:36.257073 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69978a-cff3-43f7-b89d-f09acf041241 admin admin] REQUEST_METHOD: `PUT` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.257495 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69978a-cff3-43f7-b89d-f09acf041241 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.257907 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69978a-cff3-43f7-b89d-f09acf041241 admin admin] PATH_INFO: `/v3/projects/b1179003b3194c7f915b0b8e89a95045/users/0d695d419e774320a792b5301e20db5e/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.279752 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69978a-cff3-43f7-b89d-f09acf041241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1179003b3194c7f915b0b8e89a95045, user_id=0d695d419e774320a792b5301e20db5e, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:36.282666 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69978a-cff3-43f7-b89d-f09acf041241 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.348819 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1472/2948] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:36 2022] PUT /identity/v3/projects/b1179003b3194c7f915b0b8e89a95045/users/0d695d419e774320a792b5301e20db5e/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:36.359843 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f638532-18f5-423c-b059-a799980a8174 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.360163 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f638532-18f5-423c-b059-a799980a8174 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.360436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f638532-18f5-423c-b059-a799980a8174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.387772 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1f638532-18f5-423c-b059-a799980a8174 None None] MFA Rules not processed for user `0d695d419e774320a792b5301e20db5e`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:36.389050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1f638532-18f5-423c-b059-a799980a8174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:36.454931 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1477/2949] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:36.459927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69978a-cff3-43f7-b89d-f09acf041241 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.493868 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:36.521438 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69978a-cff3-43f7-b89d-f09acf041241 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:36.526330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb6b94-b860-4206-ac48-08cd5a7b23eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:36.527156 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6b94-b860-4206-ac48-08cd5a7b23eb admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.528552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6b94-b860-4206-ac48-08cd5a7b23eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.528839 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6b94-b860-4206-ac48-08cd5a7b23eb admin admin] PATH_INFO: `/v3/projects` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.529514 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb6b94-b860-4206-ac48-08cd5a7b23eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:36.537959 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb6b94-b860-4206-ac48-08cd5a7b23eb admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.682604 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1473/2950] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:36 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:39:36.688760 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f638532-18f5-423c-b059-a799980a8174 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.699238 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f638532-18f5-423c-b059-a799980a8174 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:36.755700 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a3ee-4280-4991-afed-d7a197e5574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:36.756322 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a3ee-4280-4991-afed-d7a197e5574c admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.756592 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a3ee-4280-4991-afed-d7a197e5574c admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.756867 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404a3ee-4280-4991-afed-d7a197e5574c admin admin] PATH_INFO: `/v3/users` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.757517 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404a3ee-4280-4991-afed-d7a197e5574c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:36.759468 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404a3ee-4280-4991-afed-d7a197e5574c admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.829997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1478/2951] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:36 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:36.834157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb6b94-b860-4206-ac48-08cd5a7b23eb admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.845202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb6b94-b860-4206-ac48-08cd5a7b23eb admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:36.850002 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133150f-ffa4-42dc-8597-32e9f9b93d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:36.850648 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133150f-ffa4-42dc-8597-32e9f9b93d00 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.850927 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133150f-ffa4-42dc-8597-32e9f9b93d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.851194 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133150f-ffa4-42dc-8597-32e9f9b93d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.851845 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133150f-ffa4-42dc-8597-32e9f9b93d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:36.853474 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133150f-ffa4-42dc-8597-32e9f9b93d00 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.863449 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1474/2952] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:36.868993 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a3ee-4280-4991-afed-d7a197e5574c admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.879771 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404a3ee-4280-4991-afed-d7a197e5574c admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:36.884425 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a42ba-8271-47e9-b874-237d85a69124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:36.885069 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a42ba-8271-47e9-b874-237d85a69124 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:36.885349 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a42ba-8271-47e9-b874-237d85a69124 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:36.885626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a42ba-8271-47e9-b874-237d85a69124 admin admin] PATH_INFO: `/v3/projects/3c37709826fb4a5992541f74ae613409/users/795abc667bed4c8d9d678cf65045c132/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:36.900950 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a42ba-8271-47e9-b874-237d85a69124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c37709826fb4a5992541f74ae613409, user_id=795abc667bed4c8d9d678cf65045c132, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:36.904704 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a42ba-8271-47e9-b874-237d85a69124 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:36.961384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1479/2953] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:36 2022] PUT /identity/v3/projects/3c37709826fb4a5992541f74ae613409/users/795abc667bed4c8d9d678cf65045c132/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:36.966181 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133150f-ffa4-42dc-8597-32e9f9b93d00 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:36.976150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133150f-ffa4-42dc-8597-32e9f9b93d00 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:37.032216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853caaf-c836-4017-8a2d-23db54600e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:37.032817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853caaf-c836-4017-8a2d-23db54600e15 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:37.033116 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853caaf-c836-4017-8a2d-23db54600e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:37.033388 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853caaf-c836-4017-8a2d-23db54600e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:37.034027 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853caaf-c836-4017-8a2d-23db54600e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:37.035662 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853caaf-c836-4017-8a2d-23db54600e15 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:37.045369 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1475/2954] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:37.049584 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a42ba-8271-47e9-b874-237d85a69124 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:37.060473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a42ba-8271-47e9-b874-237d85a69124 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:37.065087 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:37.065725 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:37.066005 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:37.066315 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] PATH_INFO: `/v3/projects/3c37709826fb4a5992541f74ae613409/users/795abc667bed4c8d9d678cf65045c132/roles/97f11873550e41dba0cd095afb1dec00` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:37.068216 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c37709826fb4a5992541f74ae613409, user_id=795abc667bed4c8d9d678cf65045c132, role_id=97f11873550e41dba0cd095afb1dec00)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:37.070886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:37.081578 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1480/2955] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:37 2022] PUT /identity/v3/projects/3c37709826fb4a5992541f74ae613409/users/795abc667bed4c8d9d678cf65045c132/roles/97f11873550e41dba0cd095afb1dec00 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:37.091587 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718d9c-fc00-494e-b57e-e9df4a13e4a0 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:37.091886 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718d9c-fc00-494e-b57e-e9df4a13e4a0 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:37.092270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718d9c-fc00-494e-b57e-e9df4a13e4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:37.125828 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-c7718d9c-fc00-494e-b57e-e9df4a13e4a0 None None] MFA Rules not processed for user `795abc667bed4c8d9d678cf65045c132`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:37.127097 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c7718d9c-fc00-494e-b57e-e9df4a13e4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:37.195404 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1476/2956] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:37.207086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:37.208810 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:37.219766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:37.266899 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b727c7-e1b3-4557-a836-2a7f95db1701 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:37.270509 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebebee3-36ae-4937-99b4-fefe63749407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:37.271111 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebebee3-36ae-4937-99b4-fefe63749407 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:37.271392 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebebee3-36ae-4937-99b4-fefe63749407 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:37.271666 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebebee3-36ae-4937-99b4-fefe63749407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:37.327450 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebebee3-36ae-4937-99b4-fefe63749407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:37.328876 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebebee3-36ae-4937-99b4-fefe63749407 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:37.337643 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1481/2957] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:39.542720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7718d9c-fc00-494e-b57e-e9df4a13e4a0 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:39.544121 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-c7718d9c-fc00-494e-b57e-e9df4a13e4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:39.559826 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7718d9c-fc00-494e-b57e-e9df4a13e4a0 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:39.565493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf05b2a-e0eb-46a1-a7c2-28e4bacb2eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:39.566114 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf05b2a-e0eb-46a1-a7c2-28e4bacb2eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:39.566425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf05b2a-e0eb-46a1-a7c2-28e4bacb2eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:39.566700 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf05b2a-e0eb-46a1-a7c2-28e4bacb2eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:39.592240 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:39.625167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf05b2a-e0eb-46a1-a7c2-28e4bacb2eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:39.626642 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf05b2a-e0eb-46a1-a7c2-28e4bacb2eb5 service cinder] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:39.635586 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1477/2958] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:41.172316 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebebee3-36ae-4937-99b4-fefe63749407 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:41.181048 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebebee3-36ae-4937-99b4-fefe63749407 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:41.184651 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c418d99-e547-422b-bdcc-c6aaac844097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:41.185235 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c418d99-e547-422b-bdcc-c6aaac844097 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:41.185510 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c418d99-e547-422b-bdcc-c6aaac844097 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:41.185776 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c418d99-e547-422b-bdcc-c6aaac844097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:41.194809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c418d99-e547-422b-bdcc-c6aaac844097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:41.196214 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c418d99-e547-422b-bdcc-c6aaac844097 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:41.205811 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1482/2959] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:45.971841 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf05b2a-e0eb-46a1-a7c2-28e4bacb2eb5 service cinder] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:46.051747 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf05b2a-e0eb-46a1-a7c2-28e4bacb2eb5 service cinder] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:46.056316 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:46.056939 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:46.057218 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:46.057490 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] PATH_INFO: `/v3/users/a60e6bb2b1da4de89ec1f3d1db82d973` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:46.058625 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60e6bb2b1da4de89ec1f3d1db82d973)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:46.060541 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:46.267135 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60e6bb2b1da4de89ec1f3d1db82d973'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:46.338374 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60e6bb2b1da4de89ec1f3d1db82d973'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:46.338988 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60e6bb2b1da4de89ec1f3d1db82d973'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:46.339590 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60e6bb2b1da4de89ec1f3d1db82d973'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:46.347678 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60e6bb2b1da4de89ec1f3d1db82d973'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:46.366255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1478/2960] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:45 2022] DELETE /identity/v3/users/a60e6bb2b1da4de89ec1f3d1db82d973 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:46.370944 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebffbfc-5b09-4160-ba8b-1e845fc496b8 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:46.374159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebffbfc-5b09-4160-ba8b-1e845fc496b8 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:46.374159 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebffbfc-5b09-4160-ba8b-1e845fc496b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:46.403805 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-8ebffbfc-5b09-4160-ba8b-1e845fc496b8 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:46.409063 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8ebffbfc-5b09-4160-ba8b-1e845fc496b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:46.462507 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1483/2961] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:46.473365 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:46.477811 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:46.507064 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:46.534295 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc6346-58d4-46d4-86c6-c0b9e096506c admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:46.537885 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf50b-2d4b-4c5a-bcd5-f0db0b94038e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:46.538513 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf50b-2d4b-4c5a-bcd5-f0db0b94038e service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:46.540855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf50b-2d4b-4c5a-bcd5-f0db0b94038e service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:46.540855 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf50b-2d4b-4c5a-bcd5-f0db0b94038e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:46.548592 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbf50b-2d4b-4c5a-bcd5-f0db0b94038e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:46.550011 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbf50b-2d4b-4c5a-bcd5-f0db0b94038e service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:46.559542 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1479/2962] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:47.008663 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebffbfc-5b09-4160-ba8b-1e845fc496b8 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:47.010213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-8ebffbfc-5b09-4160-ba8b-1e845fc496b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:47.025664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebffbfc-5b09-4160-ba8b-1e845fc496b8 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:47.032814 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:47.033385 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:47.033606 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:47.033908 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] PATH_INFO: `/v3/projects/41599d2bcea240d78c26d0f262cde0bf` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:47.035143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41599d2bcea240d78c26d0f262cde0bf)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:47.037103 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:47.218634 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:47.230395 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41599d2bcea240d78c26d0f262cde0bf', 'request_id': 'req-1269ab05-f1d4-445c-a850-09d5863b42b2'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.237766 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41599d2bcea240d78c26d0f262cde0bf', 'request_id': 'req-1269ab05-f1d4-445c-a850-09d5863b42b2'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.362480 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41599d2bcea240d78c26d0f262cde0bf', 'request_id': 'req-1269ab05-f1d4-445c-a850-09d5863b42b2'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.373423 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] The token cache is being invalidate because project 41599d2bcea240d78c26d0f262cde0bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:47.373998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.374366 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.376511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1484/2963] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:47 2022] DELETE /identity/v3/projects/41599d2bcea240d78c26d0f262cde0bf => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:47.380835 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf50b-2d4b-4c5a-bcd5-f0db0b94038e service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:47.391341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf50b-2d4b-4c5a-bcd5-f0db0b94038e service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:47.450241 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:47.450873 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:47.451151 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:47.451430 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] PATH_INFO: `/v3/users/0d695d419e774320a792b5301e20db5e` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:47.452556 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d695d419e774320a792b5301e20db5e)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:47.454509 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:47.683994 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d695d419e774320a792b5301e20db5e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.790688 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d695d419e774320a792b5301e20db5e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.791303 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d695d419e774320a792b5301e20db5e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.791906 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d695d419e774320a792b5301e20db5e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.800424 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d695d419e774320a792b5301e20db5e'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:47.818491 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1480/2964] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:47 2022] DELETE /identity/v3/users/0d695d419e774320a792b5301e20db5e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:48.252823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:48.254449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:48.270502 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269ab05-f1d4-445c-a850-09d5863b42b2 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:48.327192 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:48.327824 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:48.328100 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:48.328372 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] PATH_INFO: `/v3/projects/b1179003b3194c7f915b0b8e89a95045` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:48.329493 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1179003b3194c7f915b0b8e89a95045)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:48.331439 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:48.480869 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1179003b3194c7f915b0b8e89a95045', 'request_id': 'req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.487980 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1179003b3194c7f915b0b8e89a95045', 'request_id': 'req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.567685 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1179003b3194c7f915b0b8e89a95045', 'request_id': 'req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.579231 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] The token cache is being invalidate because project b1179003b3194c7f915b0b8e89a95045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:48.579998 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.580190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.581672 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1485/2965] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:48 2022] DELETE /identity/v3/projects/b1179003b3194c7f915b0b8e89a95045 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:48.585771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:48.587282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:48.600781 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883071db-6428-424f-820c-bce1400da6cf admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:48.648076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:48.660597 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:48.661242 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:48.661521 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:48.661786 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] PATH_INFO: `/v3/users/795abc667bed4c8d9d678cf65045c132` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:48.663010 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=795abc667bed4c8d9d678cf65045c132)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:48.664921 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:48.855346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795abc667bed4c8d9d678cf65045c132'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.942552 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795abc667bed4c8d9d678cf65045c132'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.943192 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795abc667bed4c8d9d678cf65045c132'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.943806 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795abc667bed4c8d9d678cf65045c132'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.951480 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-4af0a1eb-cb85-4c8c-b41a-3fd1041df0ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795abc667bed4c8d9d678cf65045c132'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:48.970282 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1481/2966] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:39:48 2022] DELETE /identity/v3/users/795abc667bed4c8d9d678cf65045c132 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:49.380597 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:49.382410 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:49.396262 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dc871-9a71-4bdc-9e03-ea2a1b4b5106 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:49.459350 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:49.472558 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:49.473201 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:49.473483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:49.473759 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] PATH_INFO: `/v3/projects/3c37709826fb4a5992541f74ae613409` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:49.475062 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c37709826fb4a5992541f74ae613409)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:49.476994 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:49.628591 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c37709826fb4a5992541f74ae613409', 'request_id': 'req-2956cc84-914f-475e-bed8-ce3035f9abf3'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:49.635454 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c37709826fb4a5992541f74ae613409', 'request_id': 'req-2956cc84-914f-475e-bed8-ce3035f9abf3'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:49.721504 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c37709826fb4a5992541f74ae613409', 'request_id': 'req-2956cc84-914f-475e-bed8-ce3035f9abf3'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:49.732498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] The token cache is being invalidate because project 3c37709826fb4a5992541f74ae613409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:39:49.732790 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:49.733472 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-2956cc84-914f-475e-bed8-ce3035f9abf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:39:49.734992 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1486/2967] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:39:49 2022] DELETE /identity/v3/projects/3c37709826fb4a5992541f74ae613409 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:49.739796 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc546ec-4b68-4243-a875-81bd7d6c7b7e None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:49.740270 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc546ec-4b68-4243-a875-81bd7d6c7b7e None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:49.740361 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc546ec-4b68-4243-a875-81bd7d6c7b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:49.760290 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-5cc546ec-4b68-4243-a875-81bd7d6c7b7e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:49.761492 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-5cc546ec-4b68-4243-a875-81bd7d6c7b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:49.819960 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1482/2968] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:49.827897 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b7847-2786-4179-be14-e0666b57e53e None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:49.828210 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b7847-2786-4179-be14-e0666b57e53e None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:49.828489 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b7847-2786-4179-be14-e0666b57e53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:49.850228 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-1c8b7847-2786-4179-be14-e0666b57e53e None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:49.851463 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-1c8b7847-2786-4179-be14-e0666b57e53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:49.859992 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1487/2969] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:49.864453 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc546ec-4b68-4243-a875-81bd7d6c7b7e None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:49.874201 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc546ec-4b68-4243-a875-81bd7d6c7b7e None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:49.879548 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:49.880142 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:49.880420 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:49.880691 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:49.881337 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:39:49.881636 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:49.883259 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:49.895557 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1483/2970] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:39:49.900050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b7847-2786-4179-be14-e0666b57e53e None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:49.909823 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b7847-2786-4179-be14-e0666b57e53e None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:49.914632 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca3c20-1a86-44da-b76e-16d70651234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:49.915220 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca3c20-1a86-44da-b76e-16d70651234b admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:49.915495 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca3c20-1a86-44da-b76e-16d70651234b admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:49.915762 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca3c20-1a86-44da-b76e-16d70651234b admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:49.916384 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca3c20-1a86-44da-b76e-16d70651234b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:49.918340 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca3c20-1a86-44da-b76e-16d70651234b admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:50.007405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1488/2971] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:39:49 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:39:50.011584 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:50.022950 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb9d3d-ed69-4b05-a17f-221cd21772b7 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:50.073973 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:50.080898 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50a9e6-f57e-423e-8c86-89c327df7734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:50.088341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50a9e6-f57e-423e-8c86-89c327df7734 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:50.088341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50a9e6-f57e-423e-8c86-89c327df7734 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:50.088341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50a9e6-f57e-423e-8c86-89c327df7734 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:50.088341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50a9e6-f57e-423e-8c86-89c327df7734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:50.088341 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50a9e6-f57e-423e-8c86-89c327df7734 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:50.173227 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1484/2972] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:39:50 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:39:50.177995 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca3c20-1a86-44da-b76e-16d70651234b admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:50.188398 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca3c20-1a86-44da-b76e-16d70651234b admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:50.193706 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512c4a9-b8a5-47a8-bb46-baa5315ccf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:50.194422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512c4a9-b8a5-47a8-bb46-baa5315ccf62 admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:50.194707 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512c4a9-b8a5-47a8-bb46-baa5315ccf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:50.194983 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512c4a9-b8a5-47a8-bb46-baa5315ccf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:50.195626 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512c4a9-b8a5-47a8-bb46-baa5315ccf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:50.197218 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512c4a9-b8a5-47a8-bb46-baa5315ccf62 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:50.207170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1489/2973] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:50.211175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50a9e6-f57e-423e-8c86-89c327df7734 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:50.221056 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50a9e6-f57e-423e-8c86-89c327df7734 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:50.226305 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9598d-daf7-4aa4-9414-24451fa62044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:50.226899 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9598d-daf7-4aa4-9414-24451fa62044 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:50.227175 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9598d-daf7-4aa4-9414-24451fa62044 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:50.227448 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9598d-daf7-4aa4-9414-24451fa62044 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:50.228080 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9598d-daf7-4aa4-9414-24451fa62044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:50.229689 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9598d-daf7-4aa4-9414-24451fa62044 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:50.240613 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1485/2974] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:39:50.244835 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512c4a9-b8a5-47a8-bb46-baa5315ccf62 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:50.255416 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512c4a9-b8a5-47a8-bb46-baa5315ccf62 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:50.260363 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:50.260960 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:50.261240 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:50.261514 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] PATH_INFO: `/v3/projects/307bc84fa6ea41cda154754458eff813/users/08c895693aa54683951e91d4271c6191/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:50.276118 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=307bc84fa6ea41cda154754458eff813, user_id=08c895693aa54683951e91d4271c6191, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:50.279081 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:50.419164 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1490/2975] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:39:50 2022] PUT /identity/v3/projects/307bc84fa6ea41cda154754458eff813/users/08c895693aa54683951e91d4271c6191/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:39:50.425068 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d97eb-532b-4188-88c1-0958f0ed30ad None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:50.425347 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d97eb-532b-4188-88c1-0958f0ed30ad None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:50.425620 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d97eb-532b-4188-88c1-0958f0ed30ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:50.444829 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-cd4d97eb-532b-4188-88c1-0958f0ed30ad None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:50.446159 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-cd4d97eb-532b-4188-88c1-0958f0ed30ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:50.505291 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1486/2976] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:39:50.515082 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:50.516729 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:50.573862 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2648cc1-7c94-4efc-a8ce-7238c0b01827 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:50.577610 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:50.578264 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:50.578538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:50.578809 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:50.587904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:50.589345 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:50.599411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1491/2977] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:51.863644 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b8d0-3c4b-476f-9286-cced9b1d31ee None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:51.864120 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b8d0-3c4b-476f-9286-cced9b1d31ee None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:51.864216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b8d0-3c4b-476f-9286-cced9b1d31ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:51.896141 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-1e24b8d0-3c4b-476f-9286-cced9b1d31ee None None] MFA Rules not processed for user `08c895693aa54683951e91d4271c6191`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:51.897428 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-1e24b8d0-3c4b-476f-9286-cced9b1d31ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:51.967382 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1487/2978] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:51.979055 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:51.980492 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:52.007913 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:52.039061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e407d-f333-42f4-98e5-a43986b6252e service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:52.042702 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:52.043328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:52.043608 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:52.043882 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:52.055066 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:52.056538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:52.065904 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1492/2979] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:39:53.938487 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23786592-42fb-415b-95be-242c1daa0453 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:53.939044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23786592-42fb-415b-95be-242c1daa0453 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:53.939044 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23786592-42fb-415b-95be-242c1daa0453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:53.958280 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-23786592-42fb-415b-95be-242c1daa0453 None None] MFA Rules not processed for user `08c895693aa54683951e91d4271c6191`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:39:53.959471 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-23786592-42fb-415b-95be-242c1daa0453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:53.966433 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1488/2980] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:39:54.203361 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:39:54.204888 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:54.217457 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9d56c-baeb-457f-a039-7e35542755a8 service cinder] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:39:54.220968 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:39:54.221557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:39:54.221829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:39:54.222095 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:39:54.223758 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:39:54.232940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:39:54.234376 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:39:54.242683 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1493/2981] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:39:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:40:03.952297 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:04.238906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=22 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:14.443475 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e1dbd-0332-4125-a622-d28c389b8d8d None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:40:14.444047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e1dbd-0332-4125-a622-d28c389b8d8d None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:40:14.444047 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e1dbd-0332-4125-a622-d28c389b8d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:40:14.463740 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-f56e1dbd-0332-4125-a622-d28c389b8d8d None None] MFA Rules not processed for user `08c895693aa54683951e91d4271c6191`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:40:14.464949 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f56e1dbd-0332-4125-a622-d28c389b8d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:40:14.471874 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1489/2982] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:40:14.715579 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:40:14.725449 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157efe-380f-4046-a3ec-c5f9ff989c40 service neutron] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:40:14.729112 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8bbb4-6a7a-41df-8469-311f38c4fc62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:40:14.729698 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8bbb4-6a7a-41df-8469-311f38c4fc62 service neutron] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:40:14.729971 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8bbb4-6a7a-41df-8469-311f38c4fc62 service neutron] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:40:14.730279 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8bbb4-6a7a-41df-8469-311f38c4fc62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:40:14.732038 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-d2e8bbb4-6a7a-41df-8469-311f38c4fc62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:40:14.757093 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8bbb4-6a7a-41df-8469-311f38c4fc62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:40:14.758940 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8bbb4-6a7a-41df-8469-311f38c4fc62 service neutron] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:40:14.768226 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1494/2983] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:40:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:40:24.457204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:24.764199 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=11 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:33.792771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e1dbd-0332-4125-a622-d28c389b8d8d None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:40:33.794825 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-f56e1dbd-0332-4125-a622-d28c389b8d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:40:33.872506 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e1dbd-0332-4125-a622-d28c389b8d8d None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:40:33.877805 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24d70e571c64f018aa199c4c08d153e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:40:33.878454 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:40:33.878732 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:40:33.879001 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:40:33.880777 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:40:33.893710 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:40:33.895326 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:40:33.907085 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1490/2984] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:40:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:40:42.030498 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ea5a2-fc3d-4e60-892d-02687fa6d2fd None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:40:42.031003 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ea5a2-fc3d-4e60-892d-02687fa6d2fd None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:40:42.031098 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ea5a2-fc3d-4e60-892d-02687fa6d2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:40:42.091413 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-c50ea5a2-fc3d-4e60-892d-02687fa6d2fd None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:40:42.092792 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-c50ea5a2-fc3d-4e60-892d-02687fa6d2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:40:42.157319 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1495/2985] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:40:42.297618 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:40:42.299076 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:40:42.396450 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa3293-875e-4abf-b2e0-7e6151fc0211 service nova] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:40:42.401922 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb7117-2311-466b-b190-c92e16e40602 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:40:42.402555 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb7117-2311-466b-b190-c92e16e40602 service swift] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:40:42.402836 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb7117-2311-466b-b190-c92e16e40602 service swift] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:40:42.403106 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb7117-2311-466b-b190-c92e16e40602 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:40:42.411504 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb7117-2311-466b-b190-c92e16e40602 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:40:42.412970 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb7117-2311-466b-b190-c92e16e40602 service swift] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:40:42.421562 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1491/2986] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:40:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:40:52.154061 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=44 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:52.418330 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=75 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:41:00.184074 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66baca-1547-427d-9756-285e4889f58b None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:00.184557 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66baca-1547-427d-9756-285e4889f58b None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:00.184746 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66baca-1547-427d-9756-285e4889f58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:00.204114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-fa66baca-1547-427d-9756-285e4889f58b None None] MFA Rules not processed for user `df727be6307948c49affa47e601f989c`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:41:00.205317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fa66baca-1547-427d-9756-285e4889f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:00.212595 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1496/2987] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:41:00.224821 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb7117-2311-466b-b190-c92e16e40602 service swift] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:00.235931 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb7117-2311-466b-b190-c92e16e40602 service swift] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:00.240895 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94e58e-ce71-4f2c-baed-9c778c7c8e2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136ce464d3194f98952beed2d89b8bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:00.241493 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94e58e-ce71-4f2c-baed-9c778c7c8e2f service swift] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:00.241778 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94e58e-ce71-4f2c-baed-9c778c7c8e2f service swift] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:00.242083 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94e58e-ce71-4f2c-baed-9c778c7c8e2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:00.251052 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94e58e-ce71-4f2c-baed-9c778c7c8e2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:00.252632 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94e58e-ce71-4f2c-baed-9c778c7c8e2f service swift] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:00.260881 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1492/2988] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 14:41:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:41:03.404676 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66baca-1547-427d-9756-285e4889f58b None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:03.406417 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-fa66baca-1547-427d-9756-285e4889f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:03.478379 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66baca-1547-427d-9756-285e4889f58b None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:03.483513 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791154ab-e101-4745-a017-a07099a00544 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:03.484114 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791154ab-e101-4745-a017-a07099a00544 service placement] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:03.484391 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791154ab-e101-4745-a017-a07099a00544 service placement] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:03.484664 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791154ab-e101-4745-a017-a07099a00544 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:03.486455 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-791154ab-e101-4745-a017-a07099a00544 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:03.498614 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791154ab-e101-4745-a017-a07099a00544 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:03.500188 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791154ab-e101-4745-a017-a07099a00544 service placement] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:03.509999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1497/2989] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:41:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:41:03.565014 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94e58e-ce71-4f2c-baed-9c778c7c8e2f service swift] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:03.576099 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94e58e-ce71-4f2c-baed-9c778c7c8e2f service swift] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:03.580741 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:03.581370 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:03.581652 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:03.581929 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] PATH_INFO: `/v3/users/08c895693aa54683951e91d4271c6191` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:03.583132 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c895693aa54683951e91d4271c6191)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:03.585057 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:03.794202 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c895693aa54683951e91d4271c6191'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:03.871061 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c895693aa54683951e91d4271c6191'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:03.871668 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c895693aa54683951e91d4271c6191'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:03.872266 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c895693aa54683951e91d4271c6191'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:03.879792 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-9fd426b4-7426-4a01-8bdc-cbe83fb166c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c895693aa54683951e91d4271c6191'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:03.897437 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1493/2990] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:41:03 2022] DELETE /identity/v3/users/08c895693aa54683951e91d4271c6191 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:41:04.146432 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791154ab-e101-4745-a017-a07099a00544 service placement] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:04.159397 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791154ab-e101-4745-a017-a07099a00544 service placement] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:04.219143 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:04.219765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:04.220050 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:04.220328 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] PATH_INFO: `/v3/projects/307bc84fa6ea41cda154754458eff813` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:04.221466 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=307bc84fa6ea41cda154754458eff813)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:04.223418 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:04.352086 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307bc84fa6ea41cda154754458eff813', 'request_id': 'req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:04.358831 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307bc84fa6ea41cda154754458eff813', 'request_id': 'req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:04.418912 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307bc84fa6ea41cda154754458eff813', 'request_id': 'req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:04.429886 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] The token cache is being invalidate because project 307bc84fa6ea41cda154754458eff813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:41:04.430206 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:04.436213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-d01b44ef-b074-4e11-bef8-a35a91d1ca1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:04.436213 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1498/2991] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:41:04 2022] DELETE /identity/v3/projects/307bc84fa6ea41cda154754458eff813 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:41:04.437157 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7ef21-20da-4200-aa28-032411904976 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:04.437447 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7ef21-20da-4200-aa28-032411904976 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:04.437720 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7ef21-20da-4200-aa28-032411904976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:04.494969 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-62b7ef21-20da-4200-aa28-032411904976 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:41:04.496188 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-62b7ef21-20da-4200-aa28-032411904976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:04.554859 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1494/2992] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:41:04.562627 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff5a84-6b14-4d55-b415-43afb7a06dd3 None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:04.562906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff5a84-6b14-4d55-b415-43afb7a06dd3 None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:04.563175 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff5a84-6b14-4d55-b415-43afb7a06dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:04.583732 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-22ff5a84-6b14-4d55-b415-43afb7a06dd3 None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:41:04.584938 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-22ff5a84-6b14-4d55-b415-43afb7a06dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:04.593715 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1499/2993] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:41:04.598213 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7ef21-20da-4200-aa28-032411904976 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:04.608739 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7ef21-20da-4200-aa28-032411904976 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:04.613360 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:04.613962 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:04.614306 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:04.614585 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] PATH_INFO: `/v3/domains` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:04.615239 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:41:04.615546 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:04.617204 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:04.629392 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1495/2994] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 14:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:41:04.633740 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff5a84-6b14-4d55-b415-43afb7a06dd3 None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:04.643997 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff5a84-6b14-4d55-b415-43afb7a06dd3 None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:04.648829 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96af864-0a8a-4999-aa3c-1aa196d3b89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:04.649430 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96af864-0a8a-4999-aa3c-1aa196d3b89a admin admin] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:04.649712 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96af864-0a8a-4999-aa3c-1aa196d3b89a admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:04.650017 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96af864-0a8a-4999-aa3c-1aa196d3b89a admin admin] PATH_INFO: `/v3/projects` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:04.651280 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96af864-0a8a-4999-aa3c-1aa196d3b89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:04.653190 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96af864-0a8a-4999-aa3c-1aa196d3b89a admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:04.657807 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:41:04.758317 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1500/2995] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 14:41:04 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:41:04.762992 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:04.773216 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e41c4-03b7-499c-833e-6b4a3415e5ca admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:04.786037 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:41:04.831345 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97b1e-2db0-478c-a090-b6c877c4e645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:04.831952 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97b1e-2db0-478c-a090-b6c877c4e645 admin admin] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:04.832232 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97b1e-2db0-478c-a090-b6c877c4e645 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:04.832503 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97b1e-2db0-478c-a090-b6c877c4e645 admin admin] PATH_INFO: `/v3/users` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:04.833129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97b1e-2db0-478c-a090-b6c877c4e645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:04.835147 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97b1e-2db0-478c-a090-b6c877c4e645 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:04.951150 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1496/2996] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 14:41:04 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:41:04.955368 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96af864-0a8a-4999-aa3c-1aa196d3b89a admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:04.966548 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96af864-0a8a-4999-aa3c-1aa196d3b89a admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:04.971261 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3dead-b587-4af0-b445-7acbbff51e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:04.971868 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3dead-b587-4af0-b445-7acbbff51e4e admin admin] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:04.972170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3dead-b587-4af0-b445-7acbbff51e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:04.972465 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3dead-b587-4af0-b445-7acbbff51e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:04.973178 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3dead-b587-4af0-b445-7acbbff51e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:04.974917 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3dead-b587-4af0-b445-7acbbff51e4e admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:04.985115 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1501/2997] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:41:04.989407 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97b1e-2db0-478c-a090-b6c877c4e645 admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:05.000045 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97b1e-2db0-478c-a090-b6c877c4e645 admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:05.006961 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f087bfb-7f26-43c8-9c95-48957fc221a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:05.012701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f087bfb-7f26-43c8-9c95-48957fc221a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:05.012701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f087bfb-7f26-43c8-9c95-48957fc221a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:05.012701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f087bfb-7f26-43c8-9c95-48957fc221a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:05.012701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f087bfb-7f26-43c8-9c95-48957fc221a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:05.012701 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f087bfb-7f26-43c8-9c95-48957fc221a9 admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:05.029578 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1497/2998] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 14:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:41:05.034765 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3dead-b587-4af0-b445-7acbbff51e4e admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:05.045586 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3dead-b587-4af0-b445-7acbbff51e4e admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:05.051476 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:05.052153 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:05.052438 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:05.052718 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] PATH_INFO: `/v3/projects/6a6c5c4b89074bbf8ed71ef917143f14/users/5d0de576b7a74733bf3202b35391c6d2/roles/1881f748eade41d6aa06dc2097202f09` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:05.070722 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6c5c4b89074bbf8ed71ef917143f14, user_id=5d0de576b7a74733bf3202b35391c6d2, role_id=1881f748eade41d6aa06dc2097202f09)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:05.073381 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:05.154812 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1502/2999] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 14:41:05 2022] PUT /identity/v3/projects/6a6c5c4b89074bbf8ed71ef917143f14/users/5d0de576b7a74733bf3202b35391c6d2/roles/1881f748eade41d6aa06dc2097202f09 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:41:05.165425 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179053f7-bbc2-4077-a95d-2d8acae2a895 None None] REQUEST_METHOD: `POST` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:05.165713 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179053f7-bbc2-4077-a95d-2d8acae2a895 None None] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:05.165986 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179053f7-bbc2-4077-a95d-2d8acae2a895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:05.194909 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.auth.core [None req-179053f7-bbc2-4077-a95d-2d8acae2a895 None None] MFA Rules not processed for user `5d0de576b7a74733bf3202b35391c6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:41:05.196004 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-179053f7-bbc2-4077-a95d-2d8acae2a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:05.260167 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1498/3000] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:41:05.271405 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:05.273298 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:05.331777 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0ff1b-2342-41f4-8d1e-c1a1f70015e5 admin admin] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:05.335511 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368c4f5-595c-4482-bdcf-7f40a4b3e9d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '817a190d001b40a2afee944f19270aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:05.336411 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368c4f5-595c-4482-bdcf-7f40a4b3e9d9 service cinder] REQUEST_METHOD: `GET` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:05.336922 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368c4f5-595c-4482-bdcf-7f40a4b3e9d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:05.337436 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368c4f5-595c-4482-bdcf-7f40a4b3e9d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:05.347987 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368c4f5-595c-4482-bdcf-7f40a4b3e9d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:05.349473 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368c4f5-595c-4482-bdcf-7f40a4b3e9d9 service cinder] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:05.359170 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1503/3001] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:41:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:41:15.257292 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:41:15.354505 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:41:16.917817 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179053f7-bbc2-4077-a95d-2d8acae2a895 None None] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:16.982598 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179053f7-bbc2-4077-a95d-2d8acae2a895 None None] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:16.987077 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:16.987942 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:16.988087 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:16.988255 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] PATH_INFO: `/v3/users/5d0de576b7a74733bf3202b35391c6d2` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:16.989372 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0de576b7a74733bf3202b35391c6d2)` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:16.991327 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:17.251928 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0de576b7a74733bf3202b35391c6d2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:17.333860 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0de576b7a74733bf3202b35391c6d2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:17.334501 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0de576b7a74733bf3202b35391c6d2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:17.335137 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0de576b7a74733bf3202b35391c6d2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:17.342980 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.notifications [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0de576b7a74733bf3202b35391c6d2'} {{(pid=88255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:17.360228 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1499/3002] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 14:41:16 2022] DELETE /identity/v3/users/5d0de576b7a74733bf3202b35391c6d2 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:41:17.364879 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f9b3-50c2-465a-b718-f0abf36b040a None None] REQUEST_METHOD: `POST` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:17.365344 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f9b3-50c2-465a-b718-f0abf36b040a None None] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:17.365483 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f9b3-50c2-465a-b718-f0abf36b040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:17.385503 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.auth.core [None req-4364f9b3-50c2-465a-b718-f0abf36b040a None None] MFA Rules not processed for user `4c5c0285558342f3b0682eed94f7ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:41:17.386742 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4364f9b3-50c2-465a-b718-f0abf36b040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:17.448047 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1504/3003] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 14:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:41:17.458711 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:17.460124 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:17.519162 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce932e-e89f-4d50-bd6b-18b1ed12133d admin admin] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:17.523084 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a71d353c1bd849028930261e9306fa01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:17.523692 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:17.523977 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:17.524244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:17.533771 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:17.535307 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:17.545410 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1500/3004] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:41:17.975620 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364f9b3-50c2-465a-b718-f0abf36b040a None None] Authenticating user token {{(pid=88256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:41:17.977615 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.fernet_utils [None req-4364f9b3-50c2-465a-b718-f0abf36b040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:41:17.991601 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364f9b3-50c2-465a-b718-f0abf36b040a None None] Validating token access rules against request {{(pid=88256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:41:17.996431 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c5c0285558342f3b0682eed94f7ff4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '273637bb45fb4aad868a7d2db9ada393', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:41:17.997137 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:41:17.997422 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:41:17.998331 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] PATH_INFO: `/v3/projects/6a6c5c4b89074bbf8ed71ef917143f14` {{(pid=88256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:41:17.999538 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6c5c4b89074bbf8ed71ef917143f14)` {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:41:18.001516 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] RBAC: Authorization granted {{(pid=88256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:41:18.118239 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6c5c4b89074bbf8ed71ef917143f14', 'request_id': 'req-efb2d63a-4537-43c8-ad5d-8ddad9844de7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:18.125102 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6c5c4b89074bbf8ed71ef917143f14', 'request_id': 'req-efb2d63a-4537-43c8-ad5d-8ddad9844de7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:18.208906 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6c5c4b89074bbf8ed71ef917143f14', 'request_id': 'req-efb2d63a-4537-43c8-ad5d-8ddad9844de7'} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:18.222799 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] The token cache is being invalidate because project 6a6c5c4b89074bbf8ed71ef917143f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:41:18.222999 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:18.223710 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG keystone.notifications [None req-efb2d63a-4537-43c8-ad5d-8ddad9844de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:41:18.225223 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: [pid: 88256|app: 0|req: 1505/3005] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 14:41:17 2022] DELETE /identity/v3/projects/6a6c5c4b89074bbf8ed71ef917143f14 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:41:27.540672 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:41:28.218915 dv-cinder-856460-1-iscsi devstack@keystone.service[88256]: DEBUG dbcounter [-] [88256] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:43:24.757911 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] Authenticating user token {{(pid=88255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:43:24.759838 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:43:24.878109 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd7c9d-0f7e-4689-8ea2-ad7e0bbe9501 service neutron] Validating token access rules against request {{(pid=88255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:43:24.939264 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77146f24-6991-426d-9280-ac1a1232d846 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '645282ae0ac7450dbd27d71c467b5216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e708e964ac564595b5abab68cbea89e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:43:24.939914 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77146f24-6991-426d-9280-ac1a1232d846 service placement] REQUEST_METHOD: `GET` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:43:24.940233 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77146f24-6991-426d-9280-ac1a1232d846 service placement] SCRIPT_NAME: `/identity` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:43:24.940533 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77146f24-6991-426d-9280-ac1a1232d846 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:43:24.942402 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.fernet_utils [None req-77146f24-6991-426d-9280-ac1a1232d846 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:43:24.954756 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77146f24-6991-426d-9280-ac1a1232d846 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:43:24.956346 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77146f24-6991-426d-9280-ac1a1232d846 service placement] RBAC: Authorization granted {{(pid=88255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:43:24.967129 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: [pid: 88255|app: 0|req: 1501/3006] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 14:43:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:43:34.962244 dv-cinder-856460-1-iscsi devstack@keystone.service[88255]: DEBUG dbcounter [-] [88255] Writing DB stats keystone:SELECT=78 {{(pid=88255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}